http://invisible-island.net/ncurses/ncurses.faq.html
Copyright © 2017-2022,2024 by Thomas E. Dickey

Here is the latest version of this file.

NCURSES – comments on S-Lang

(FAQ)
(top)
Synopsis
Disclaimer
Tie-in with ncurses
Symbols used by tie-ins
- curses/ncurses symbols
- slang symbols
General discussion
Influences
Development timeline
Documentation
Source archives
Followup

Synopsis

Here is a discussion of the S-Lang library (better known as slang) and its relationship versus ncurses.

Disclaimer

I do not use S-Lang. This is why:

Early on (beginning in Fall 1994), the most pager looked interesting. I was collecting tools for my $dayjob, and thought others on my project might find it useful.

To a lesser extent, slrn was interesting because I wanted to compare it with tin.

The slang library had problems; the applications built with it crashed. I reported these bugs to John Davis, pointing to places that Purify identified as problems. Those were not addressed; he was unable to see the problem. The last one that I reported was fairly obvious since the program was attempting to store a pathname longer than 80 characters in an 80-character buffer (for the title line).
Besides the lack of progress on bug reports, there was an extended (5-month) interval in 1996 where most would not compile with the current library.

In his email reply, John Davis said only

I need to update most to compile with recent versions of slang.

This was not an isolated incident (see comments on compatibility).

As part of the tool-collection activity, I went to an ftp site where I had found sc (a curses-based spreadsheet program published in comp.sources.misc, volumes 20, 22, 23, 30 in 1991–1992), and did not find it. I noticed slsc in the listing and contacted the site owner, who replied that they'd removed sc because slsc was said to be the latest version of sc, and that there was no reason to have the older version.

slsc, of course was not the latest version. Davis made a copy of sc and changed all of the key bindings. This was a few years before Davis produced any documentation. Quoting from the README file:

I cannot possibly list all of the changes that I have made to the program.
It is my guess that the size of the diffs are greater than the source itself.
Previously, sc relied on curses for screen management.  There were comments
in the code suggesting that curses is buggy.  Whether it is or not, I did
away with curses and incorporated S-Lang's screen management routines.  In
addition, I am using S-Lang's ability to handle keymaps, readline, help
routines etc...  For these reason's I have changed the name to `slsc'.   The
upshot is that it is a new program with a new name, and a new look and feel.

Please note that there is no man page for slsc yet.  sc.doc is the man page
for the version of sc that slsc is based on.

Given all of that, there was no basis for collaboration. There were other problems:

Davis does not use dates in his changelogs. If someone's going to tell me what they did, but not when, they can revise their story to fit whatever need they have at the moment.
His changelog in the 1990s did not mention contributors until September 1996 (after four years of work). I had enough of that with ncurses.

Tie-in with ncurses

So much for background. The reader may ask what this has to do with ncurses. The answer is that the slang library provides terminal display features something like curses. In promoting the use of his library, this has been the main application area for it.

Early on (probably 1995 or 1996 — some reader may find this in a usenet archive),

Davis made a statement to the effect that slang was better than curses because slang supports color and curses does not.
I noticed the discussion and replied that he was probably basing his statement on his familiarity with (low-budget) academic systems which tended to lag several years behind the (high-priced, licensed) systems where curses development had been for the past decade.
Davis replied that he was using the very newest curses, and that no, it did not support color.
I mentioned that ncurses supported color.
Davis replied that ncurses was not curses.

Having been properly introduced, we continued on in that vein for another ten years.

Besides discussion, we both provided changes for other programs. Only a few programs have been modified to easily build/work with either library.

Only a few well-known applications have been developed which could use either ncurses or slang. I have been involved with Lynx, Mutt and Midnight Commander. Most of the early discussion mentions the possibility of others, but provides no useful examples.

Davis' initial comments mentioned only those applications (using slang of course) which he had written, modified or otherwise contributed to. Responding to a thread EDT compatible editor on comp.editors in 1995, he wrote:

On 27 Jul 1995 12:06:39 GMT, Ulli Horlacher
<zrxh...@baracke.rus.uni-stuttgart.de> wrote:
 : very powerfull. JED has its own news group: alt.lang.s-lang

It should be pointed out that jed embeds an interpreter called `s-lang'.
Since jed is the most well known application that utilizes s-lang, almost
every article in alt.lang.s-lang pertains to jed.

There are other applications that utilize the s-lang library.  These include:

   slrn  (This newsreader)
   slsc  (Spreadsheet)
   dosemu (DOS emulator for linux)

as well as several other applications that are not in circulation.  In
addition, I know of at least two other programs that are in wide circulation
and are making the move from curses to slang.  I think it is reasonable to
expect articles in alt.lang.s-lang pretaining to any of these applications.

--John

Possibly he was referring to Midnight Commander as one of the two programs; the timeframe for the other two does not work for his comment since he got involved with Lynx and Mutt in 1996.

Some of Davis' enthusiasm for his own work influenced others, such as Larry Ayers. In an article a year later in the Linux Gazette, Ayers mentioned Minicom, crediting slang for its use of color. That may have been the second program which Davis had in mind.

I reported problems with Minicom, and sent fixes not specific to ncurses from 1995-1998, and continued using it through 1999 although I was working on xterm. For a while, I used seyon (a wrapper around xterm). I noticed Ayer's article in 1998, and sent mail pointing out some problems with it, e.g.,

Furthermore, you don't distinguish between slang-the-language and
slang-the-screen facility -- same library of course, but minicom, for
example, supported color before someone decided to hook it up with slang.

Lynx

My mail first mentions Lynx (2.4.2 built with ncurses 1.9.3) in July 1995. Later, in November, Tony Nugent mentioned that Lynx did not support color.

I subscribed to the lynx-dev mailing list in April 1996, thinking to follow up on Nugent's comments. I was busy with other projects, and initially just read the discussion on the mailing list.

In June 1996, Andrew Kuchling sent mail to Zeyd's ncurses mailing list stating that he had enabled mouse-support in Lynx (using ncurses) for xterm, and was surprised to find that it interfered with select/paste. The workaround for that, of course, would be to use the Shift key while selecting and pasting.

John Davis also was interested in Lynx. He modified Lynx to work with slang 0.99-30 on VMS, as noted in Lynx's CHANGES2.5 by Foteos Macrides:

03-18-96
* Added SLANG support (for colorized Lynx) based on patches from JED.
  The make support is for linux and sun4, but it should be portable to
  other Unix flavors.  I have it working on VMS, but the SLANG interrupt
  and exit handlers are incompatible with the current ones for VMS in
  Lynx (and with the VMS debugger), so I haven't yet included SLANG
  support in build.com or the MMS files. - FM

Later in 1996, I noticed a lynx-dev comment by Davis that once his changes were applied, Lynx would no longer build with curses. Macrides amended that, probably mentioned here (CHANGES2.7):

10-19-96
* Added definitions in LYCurses.h relating curses function calls to slang
  functions or emulations for compatibility with new versions of slang,
  based on patch from John E. Davis (davis@space.mit.edu). - FM

In between, you can find the first mention of slang's "curses-compatibility":

06-21-96
* Mods to revive clearok() functionality with the slang 0.99-33 library
  (need it for Kanji handling and for VMS), and to deal with the definition
  conflicts for the (poorly added) TRUE and FALSE definitions in the new
  slang library's SLcurses.h. - FM

Davis added a minimal slcurses.h in 1994 (seen in 0.99-7):

#include "slang.h"
#define move SLsmg_gotorc
#define clreol SLsmg_erase_eol
#define printw SLsmg_printf
#define COLS SLtt_Screen_Cols
#define LINES SLtt_Screen_Rows
#define clrtobot SLsmg_erase_eos
#define clrtoeol SLsmg_erase_eol
#define standout SLsmg_reverse_video
#define standend  SLsmg_normal_video
#define addch SLsmg_write_char
#define addstr SLsmg_write_string
#define initscr SLtt_get_terminfo (); SLsmg_init_smg
#define refresh SLsmg_refresh
#define clear SLsmg_cls
#define mvaddstr(y, x, s) SLsmg_gotorc(y, x); SLsmg_write_string(s)
#define touchline SLsmg_touch_lines
#define inch SLsmg_char_at
#define endwin SLsmg_reset_smg

to allow him to compile the worm program written by Eric Scott in 1980. That was too “minimal” to be generally useful. I implemented something like that for ded at the end of 1985, but discarded it a few months later when I moved to a more reliable development machine. Davis kept slcurses.h, adding a few more lines by 0.99-30 (April 1996):

#ifdef A_NORMAL
#undef A_NORMAL
#endif
#define A_NORMAL 0

#define scrollok(a,b) SLsmg_Newline_Moves = ((b) ? 1 : -1)
#define getyx(a,y,x)  y=SLsmg_get_row(), x=SLsmg_get_column()
#define echo()
#define noecho()
#define cbreak() SLang_init_tty (7,0,0)
#define crmode() cbreak()

and began developing a revised “slcurses” header (and supporting C-file) beginning in 0.99-32 (June 1996), culminating in 1326 lines in 0.99-38 (January 1997).

The slcurses.h file has grown since that point, but includes the same features. Interestingly enough, the 0.99-38 header file included this comment (present in the current version):

/* This is a temporary hack until lynx is fixed to not include this file. */
#ifndef LYCURSES_H

although Lynx has not included that header since October 1996.

Besides contemplating a recurrence of slsc, I could see a problem:

Lynx2.4's top-level makefile had 46 targets for the various combinations of system and library (two used slang: linux-slang and sun4-slang).
Lynx2.6 (four months later) had 62 targets for system/library (13 were for slang).

That is, as Lynx was ported to different systems, there would be at least one makefile target, multiplied by the different compilers used and again by the number of possible screen-libraries.

Addressing the configuration issue, I got involved with Lynx development in January 1997, working with Jim Spath (to determine the requirements for an autoconf configure-script). At the time, Klaus Weide set up a development area, using PRCS for source control. Foteos Macrides was uninterested in either of those developments, leaving when Lynx 2.8 was released. I replaced the makefiles with ones generated by the configure script, and made the configure script handle the quirks of ncurses and slang libraries. Thereafter, changes to Lynx had to build properly with either library.

Others disagreed, suggesting that Lynx might be modified to use SLcurses. I pointed out that was unlikely to happen in my response to the thread lynx-dev Suggestion for merging the libcurses and libslang code:

> if lynx had been slang-ified later, when just including slcurses.h would
> have provided most or all of the needed curses functions and macros
> under their curses names.

perhaps - but John Davis has stated that he prefers to not use the slcurses.h
header, but code directly with slang's functions.

When I have made changes to features where slang is used, I do (try to) ensure that I do not break something. For instance, when I revised the popup-menu feature in 2000, I filled in the pieces to keep the slang configuration working. But for new features, my policy has always been that the work will be done by others. However, others seem content to supply only build-fixes.

In practice, Lynx's support for Unicode is better with ncurses/ncursesw. Early on, Lynx had “support” for UTF-8 in the sense that in a half-dozen places in the source-code it knew about the continuation-bytes for a UTF-8 character. Its notion of the width of those characters was poor, causing problems when deciding if it had reached the right margin. After much cleanup work, Lynx does an acceptable job with ncursesw, but some of that cleanup applies only to the ncurses/ncursesw configurations.

In 2005, John Davis said he would supply an improved patch for Lynx using the new (and changed) features in slang2, but he provided no changes beyond a build-fix for 2.8.6dev.12:

* modified fix for slang 2.0 which makes it clearer that the last parameter
  of SLexpand_escaped_string is for UTF-8 -JED

Others provided build-fixes for slang2. See for example the mailing list thread [Lynx-dev] Patch for S-Lang 2.0 in February 2005:

* fixes to allow compiling with slang 2.0 prerelease 5 -GV

That “allow compiling” was necessitated by Davis having changed the datatype used for cells in his curses-wrapper (and other display functions). Programs written for slang (version 1) are not compatible with those written for slang2.

Mutt

I have been on the current mutt-dev mailing list started by Thomas Roessler since May 2000, and posted messages to Michael Elkins' earlier mutt-dev since 1997, but my involvement with Mutt started in January 1996, when Michael Elkins commented on the ncurses mailing list:

Subject: Re: handling SIGWINCH?
To: elkins@aero.org (Michael Elkins)
Date: Fri, 26 Jan 1996 05:42:02 -0500 (EST)
Cc: ncurses-list@netcom.com (Ncurses Mailing List)

> I've been writing my own MUA and using ncurses to do the UI.  One thing
> that I would like to be able to do is to automatically repaint the screen
> at it's new size whenever a user resizes an xterm.  This generates a
> SIGWINCH signal which I catch.  However, I cannot find any function in
> the ncurses library that would seem to cause it to set LINES and COLS
> correctly...  Does anyone know how I might be able to do this?
sure. It's in my resize-197 patch on sunsite.unc.edu (a function called
'resizeterm', along with the original version of wresize).

--
Thomas E. Dickey
dickey@clark.net

He was referring to Mutt, which was in an early stage of development by then. Although he used ELM for the above message, later that day he sent another message with this in the header:

X-Mailer: Mutt 0.1.6

By the way, that 0.1.6 looks odd because Mutt's version numbering changed shortly after beginning development. Reviewing the mail from early 1996, these were used for Mutt:

X-Mailer: Mutt 0.1.6
X-Mailer: Mutt 0.1.9
X-Mailer: Mutt 0.20
X-Mailer: Mutt 0.27

demonstrating that (and agreeing with the copyright notices in mutt 0.47), Elkins probably began work on Mutt in January 1996. His change-logs have no dates. Fortunately, Sven Guckes has a history page with many of the early release dates, but none go back before May 1996. For alternate facts, refer to this page.

Mutt 0.47 was released in October 1996, with a note in the changelog written few days before, reflecting Elkins' intentions for the next stage of development:

Removed all subwindow curses code.  This is in preparation for using the S-Lang
library as an alternative to ncurses.

S-Lang [sic] does not support overlapping windows (one of several impediments to porting a program from curses to slang), though one might argue the point about subwindows. The library is named slang in any case.

Regarding resizeterm, I was rather busy, and so apparently was Elkins, who continued the conversation about a year later:

Date: Tue, 12 Aug 1997 11:16:22 -0700
From: Michael Elkins <me@cs.hmc.edu>
To: ncurses@bsdi.com
Cc: mutt-dev@cs.hmc.edu
Subject: [ncurses 4.1] getch() behavior on SIGWINCH
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Mailer: Mutt 0.82
Sender: owner-ncurses@mailgate.BSDI.COM
Precedence: bulk
Status: RO

Howdy,

I'm finally getting around to getting Mutt to properly resize the screen
with ncurses.  I just use the resizeterm() function and everything seems to
work quite well.  However, I cannot get things set up to automatically
repaint the screen when the user resizes the term because getch() is not
interrupted when the SIGWINCH signal is received.  For example, when using
the slang library, getch() returns ERR and then I do a check to see if
SIGWINCH has been caught and I can repaint.  Is there any way to change the
behavior of getch() ?  Currently I have to press a key to get the screen to
repaint and that's not the desired behavior...

Thanks,
me
--
Mutt: a curses mail client for Unix systems.  http://www.cs.hmc.edu/~me/mutt/

In between, Michael Elkins had become interested in making Mutt build with either curses or slang. John Davis had started a curses-like interface for use with Lynx, but it was rudimentary. Here is an extract from slang's changes.txt:

Changes since 0.99-36
...
3. Updates to slcurses.c by Michael Elkins.
...
Changes since 0.99-35
5. Improved curses emulation provided by Michael Elkins
   (me@muddcs.cs.hmc.edu).
...
Changes since 0.99-31
1. Simple minded curses emulation added.  This should help those migrating
   from curses.  See slang/src/curses/*.c for sample curses demo programs
   that the SLsmg routines can handle.  See also slang/src/slcurses.c for
   some wrapper functions.

Judging by the dates and change-log items, Michael Elkins (who used this interface in Mutt as an alternative to curses) did most of the work. Using numbers from diffstat:

0.99-32 (Davis)

 src/slcurses.c        |  246 +++++++++
 src/slcurses.h        |  130 ++++-

0.99-34 (Elkins)

 src/slcurses.c        |  226 ++++++++++++++++++++++++++
 src/slcurses.h        |   77 ++++++++-

0.99-35 (Elkins)

 src/slcurses.c        |  557 ++++++++++++++++++++++++++++++++++++-----------
 src/slcurses.h        |  256 +++++++++++++++------

0.99-36 (Elkins)

 src/slcurses.c        |  291 +++++++++++++++++++++++++++++++-------
 src/slcurses.h        |   89 ++++++++---

Mutt's change-log after 0.47 reflects a few additional incompatibilities between ncurses and slang which Michael Elkins discovered:

[New] --with-slang configure option to use slang instead of curses/ncurses.
You MUST get ftp://space.mit.edu/pub/davis/slang0.99-35.tar.gz AND apply my
patch (ftp://cs.hmc.edu/pub/me/slang0.99-35me.diff) OR pick up a patched
copy (ftp://cs.hmc.edu/pub/me/slang0.99-35me.tar.gz)

- xterm mouse support has been removed (SLsmg and ncurses are not compatible
  as far as mouse support)

- ungetch() doesn't work with KEY_* under SLang, so use mutt_ungetch() for
  both ncurses and slang

Michael Elkins' followup in August 1997 about resizing started a discussion on the ncurses mailing list which led to Alexander V. Lukyanov suggesting the KEY_RESIZE extension along with a SIGWINCH handler. He provided those changes in September 1997.

Independently of that, I checked on reports of resizing problems with Mutt and ncurses, e.g., on FreeBSD. There were also issues with color. I sent a small patch to the Mutt developer's list early in August 1997, and got a response from Liviu Daia. Among other things, we discussed the default-colors extension, which he decided to use:

On 5 August 1997, T.E.Dickey <dickey@clark.net> wrote:

[ adding transparent colors to Mutt ]

> >     Yeah, we need to do this in order to re-use color pairs. I'll
> > try to post a patch to that effect in a day or two if I can figure
> > out what use_default_colors() does. :-)
>
> I modified a couple of the test-programs in ncurses to use this
> feature (btw the test directory can be configured against either
> ncurses or SVr4 curses). The hanoi, worm, firework programs all draw
> colors on a default background.
>
> I considered making the colors stored so that you use the color-pair
> index rather than the mask; indices are positive & would fit in with
> the existing scheme - but would still be a fairly large change.

    I finaly got the time to look at this. It turned out that the
problem was not related to storing the colors masked, but to the parsing
routine, which expect mutt_getvaluebyname() to return -1 only on error.
:-) I posted a patch to mutt-dev which works with both ncurses 4.1 and
S-Lang. It even allows "brightdefault" colors now. :-)

    Thanks again for the use_default_colors() hint!

    Regards,

    Liviu

Liviu made many improvements to Mutt, running into various incompatibilities between ncurses and slang. This chunk from pager.c (in 0.88, November 1997) illustrates the problems:

  /* avoid a bug in ncurses... */
#ifndef USE_SLANG_CURSES
  if (col == 0)
  {
    SETCOLOR (MT_COLOR_NORMAL);
    addch (' ');
  }
#endif

  /* end the last color pattern (needed by S-Lang) */
  if (last_special || (col != COLS && (flags & (M_SHOWCOLOR | M_SEARCH))))
    resolve_color (*lineInfo, n, vch, flags, 0, &a);

  /* ncurses always wraps lines when you get to the right side of the
   * screen, but S-Lang seems to only wrap if the next character is *not*
   * a newline (grr!).
   */
#ifndef USE_SLANG_CURSES
    if (col < COLS)
#endif
      addch ('\n');

Comments in source code can last long after the initial reason for the change is lost or obscured. As of August 2017, that comment (and section of code) is almost the same:

  /* avoid a bug in ncurses... */
#ifndef USE_SLANG_CURSES
  if (col == 0)
  {
    NORMAL_COLOR;
    addch (' ');
  }
#endif

  /* end the last color pattern (needed by S-Lang) */
  if (special || (col != pager_window->cols && (flags & (MUTT_SHOWCOLOR | MUTT_SEARCH))))
    resolve_color (*lineInfo, n, vch, flags, 0, &a);

  /*
   * Fill the blank space at the end of the line with the prevailing color.
   * ncurses does an implicit clrtoeol() when you do addch('\n') so we have
   * to make sure to reset the color *after* that
   */
  if (flags & MUTT_SHOWCOLOR)
  {
    m = ((*lineInfo)[n].continuation) ? ((*lineInfo)[n].syntax)[0].first : n;
    if ((*lineInfo)[m].type == MT_COLOR_HEADER)
      def_color = ((*lineInfo)[m].syntax)[0].color;
    else
      def_color = ColorDefs[ (*lineInfo)[m].type ];

    ATTRSET(def_color);
  }

  if (col < pager_window->cols)
    mutt_window_clrtoeol (pager_window);

  /*
   * reset the color back to normal.  This *must* come after the
   * clrtoeol, otherwise the color for this line will not be
   * filled to the right margin.
   */
  if (flags & MUTT_SHOWCOLOR)
    NORMAL_COLOR;

However, to me, that "avoid a bug" looks like a misunderstanding: if the window colors are set when a program prints a newline, the colors will be propagated to the next line. The comment and logic in the current program agree with that. The relevant change-log item might be this;

- various fixes to the pager hiliting [Liviu Daia <daia@stoilow.imar.ro>]

Aside from suggestions, I also sent a few fixes for problems related to curses/ncurses:

Mon Nov 15 17:44:15 1999  Thomas Roessler  <roessler@guug.de>

        * pgp.c: Fix the pgp-hook logic.

        * color.c: Small color fix from T. E. Dickey.

        * color.c: A small patch from TE Dickey.

Most of the curses-related work on Mutt was done by others, dealing with its color schemes (and making those work with ncurses or slang).

Because curses is a well-established library in many versions, and because ncurses can be configured to avoid conflict with another version of curses, there are several special cases to handle in an autoconf script. Lars Hecking used some of my autoconf macros for Mutt which are still used in 2017:

Date: Tue, 13 Jun 2000 20:42:40 +0100
From: Lars Hecking <lhecking@nmrc.ie>
To: Kasopa Wilbroad Chisanga <kasopa.w.chisanga@coppernet.zm>
Cc: mutt-users@mutt.org
Subject: Patch for better curses detection [was: Re: Seeking help]
Mail-Followup-To: Kasopa Wilbroad Chisanga <kasopa.w.chisanga@coppernet.zm>,
        mutt-users@mutt.org

> This is what stopped the build - HP has a different flavor of curses
> which should define this (curs_colr), but mutt's configure script
> doesn't recognize that (doesn't generate -I and -L options for the
> makefile to make it use the newer library).

 Try this patch. It's relative to mutt-cvs, but should work with 1.3.x,
 maybe even 1.2.x. Minor ugliness (?): I haven't changed the macro, so
 it uses LIBS instead of MUTTLIBS.

 Autogenerated files are not included, so you need the environment
 described in doc/devel-notes.txt, especially autoconf and automake.

 I hope I managed not to break anything :-)

 If this works, thank T.E.D :-))

diff -urN mutt-cvs/ChangeLog mutt-1.3.3/ChangeLog
--- mutt-cvs/ChangeLog  Sat Jun 10 06:30:29 2000
+++ mutt-1.3.3/ChangeLog        Tue Jun 13 20:17:01 2000
@@ -1,3 +1,11 @@
+2000-06-13  Lars Hecking  <lhecking@nmrc.ucc.ie>
+
diff -urN mutt-cvs/ChangeLog mutt-1.3.3/ChangeLog
--- mutt-cvs/ChangeLog  Sat Jun 10 06:30:29 2000
+++ mutt-1.3.3/ChangeLog        Tue Jun 13 20:17:01 2000
@@ -1,3 +1,11 @@
+2000-06-13  Lars Hecking  <lhecking@nmrc.ucc.ie>
+
+       * configure.in: Use CF_CURSES_LIBS macro for better detection
+       of platform-specific curses libraries.
+
+       * m4/curslib.m4: New file, contains Tom Dickey's CF_CURSES_LIBS
+       macro from lynx.
+
 Fri Jun  9 11:34:26 2000  Thomas Roessler  <roessler@guug.de>

        * po/ru.po, po/sk.po, po/sv.po, po/uk.po, po/zh_TW.Big5.po,
diff -urN mutt-cvs/configure.in mutt-1.3.3/configure.in
--- mutt-cvs/configure.in       Tue Jun 13 06:30:17 2000
+++ mutt-1.3.3/configure.in     Tue Jun 13 20:08:45 2000
@@ -183,7 +183,7 @@
                fi
                AC_CHECK_HEADERS(ncurses.h)],

-               [MUTTLIBS="$MUTTLIBS -lcurses"])
+               [CF_CURSES_LIBS])

        old_LIBS="$LIBS"
        LIBS="$LIBS $MUTTLIBS"
diff -urN mutt-cvs/m4/curslib.m4 mutt-1.3.3/m4/curslib.m4
...

For the slang configuration, the problem is simpler, since only the very latest version of that library is supported at any moment in time.

Midnight Commander

While I do use Lynx and Mutt, I do not use Midnight Commander:

First, of course, I have my own tool ded which does what I need. Midnight Commander does not.
In the mid-1990s I did take into account differing tastes of other people on my project, while collecting tools for them. At that point, Midnight Commander failed my initial screening because it relied upon gcc to build.

That was the “statement expression” feature. It is non-standard, and not everyone wants it (see for example How to Get Fired Using Switch Statements & Statement Expressions, by Robert Elder). Checking the copies of mc at hand, I see that feature used in the intl library bundled in 4.1.35 (May 25, 1998), but not in 4.1 (September 16, 1997).
I evaluated Midnight Commander in late 1997, but determining an actual date is impossible, because the developers did not maintain a usable changelog. In each version, “NEWS” file does not mention the current release, has no dates, and is too short to be a plausible record.

For example, the 4.1.35 src-directory contained ChangeLog and OChangeLog totalling 521Kb, with mention of 4.1.5 and 4.1.4 in one check-in comment related to when a bug was inserted. The NEWS file stopped at 3.5 and is 17Kb (3% of the detailed log). None of the change-logs mention when a release was made using the problematic intl library.

That being established, Midnight Commander's developers were users of ncurses. Pavel Roskin provided some useful feedback.

Before I started maintaining ncurses in 1996, Midnight Commander had for a while used ncurses. A posting to comp.os.linux.announce ANNOUNCING The Midnight Commander v2.0 (1995/02/20) said:

   You may need to have certain auxiliary packages installed on your
machine to be able to build the Midnight Commander.  They are all
available at ftp.nuclecu.unam.mx so that you can download them at the
same time as the Commander if you wish.  They are:

 * ncurses 1.8.5.2        ftp.nuclecu.unam.mx in /linux/local
                        ftp.netcom.com in /pub/zm/zmbenhal
   Notes:
      Don't get ncurses 1.8.6; it's buggy.
      If ncurses hasn't been ported to your system you can try to
      build the Commander with the native curses library on your
      system.  This works well on System V flavoured systems, but
      it may not be very pretty or even usable on BSD systems.

      If you use a version of ncurses different than 1.8.5.2, we
      include a patch for making it 8 bit clean.

 * gpm-0.97                ftp.nuclecu.unam.mx in /linux/local
                        iride.unipv.it in /pub/gpm
   Notes:
      This is the excellent Linux "General Purpose Mouse" server;
      needed if you want mouse support in Linux virtual consoles.

The 2.1 snapshot from February 1995 (which actually has a usable ChangeLog file) elaborated on that, referring to ncurses versions from 1994:

You'll need GNU C (or an ANSI C Compiler) and a color curses library
(ncurses is a good choice).  For more information on obtaining these
programs refer to the section "Obtaining the missing pieces" in this
file.

The best choice is to use ncurses 1.8.5.2 available in
ftp.nuclecu.unam.mx in the /Midnight directory.  The program can be
compiled with Ultrix's curses using the --with-bsd-curses and on SunOS
4.x SystemV curses with the --with-sunos-curses.

Many Linux systems ship with version 1.8.0 or 1.8.1; the Midnight
Commander may run with it, but some bugs in this version of NCurses
sometimes make the display look strange (which is not the intention).
Also, there is a new version 1.8.6 and 1.8.7 of ncurses, you may run
into some problems installing those versions.

So, if you have the time, give yourself a little present and install
the ncurses 1.8.5.2 library on your box.

VERY IMPORTANT: On some architectures, the default compilation mode
for ncurses it to include the buggy OPT_MVCUR in the Makefile.  If
that is the case, please remove it before compiling

but it also has a couple of lines in its TODO file hinting at a change:

* Features or problems
  - Quick, Quick, Quick: Use slang instead of ncurses.
...
* Optimizations and cleanup
...
  - Switch to the S-Lang screen manager.

The ChangeLog file gives some hint about the problem they found with the 1994-versions of ncurses: they were (as I was) attempting to use it with SIGWINCH.

Perhaps they were misled by this item in LinuxNews from early 1993 (not the only questionable item in that venue):

-- January 28 Zeyd M. Ben-Halim has released a new version of
his port of ncurses.
This version should be usable by the general public as it is more stable.
ncurses is a SYSV-compatible curses (screen management) package.
It supports keypad() allowing access to function and arrow keys.
It supports and uses Terminfo instead of termcap for terminal description.
It has support for PC graphic characters (used in drawing boxes).
It has color support similar to that in SYSV curses.
It has support for LINES and COLUMNS environment variables.
It has support for SIGWINCH.
FTP:sunsite.unc.edu:/pub/Linux/Incoming, tsx--11.mit.edu:/pub
 /linux/sources/libs/

Referring to my email,

From miguel@roxanne.nuclecu.unam.mx Fri Mar 15 09:48 EST 1996
Received: from roxanne.nuclecu.unam.mx (roxanne.nuclecu.unam.mx [132.248.29.2]) by mail.Clark.Net (8.7.3/8.6.5) with SMTP id JAA17865 for <dickey@clark.net>; Fri, 15 Mar 1996 09:48:09 -0500 (EST)
Received: (from miguel@localhost) by roxanne.nuclecu.unam.mx (8.6.12/8.6.11) id IAA12026; Fri, 15 Mar 1996 08:47:14 -0600
Date: Fri, 15 Mar 1996 08:47:14 -0600
From: Miguel de Icaza <miguel@roxanne.nuclecu.unam.mx>
Message-Id: <199603151447.IAA12026@roxanne.nuclecu.unam.mx>
To: dickey@clark.net
In-reply-to: <199603151126.GAA09909@clark.net> (dickey@clark.net)
Subject: Re: Midnight commander pr
X-Consejo: Borra Windows
Content-Type: text
Content-Length: 1457
Status: RO

Hello Thomas,

> I've noticed that the ncurses guys (zeyd & eric) are rather slow to respond.
> What sort of problems did you have (remaining) with ncurses before you gave
> up on it?  (I've been fixing or persuading them on the ones that impact me).

There were many things: at the time we took the decision on switching
to Slang, Eric was arriving to the ncurses package, so ncurses was a
mess, Zeyd did not fix the bugs on the code nor we did get any
acknoledgement from him, we even had to package a couple of fixes to
ncurses.  We would have helped, but we already had a busy schedule
with the vfs code.

We had to remove features to make it work under ncurses (for example,
from the ncurses mailing list, I know that until a couple of weeks
ago, they still had problems with programs that had their terminal
resized, ncurses would just crash).  We have had proper terminal
resizing for a while now.  If ncurses would have been as robust as it
looks now, we maybe would have stayed with them..

Now, Slang is faster than ncurses (someone posted some benchmarks some
time ago), so in the long run, we had a faster screen manager.
Anyways, we can do both ncurses and slang :-).

> (I have a directory editor which is different in design from MC, which I'm
> keeping curses-based).

Is it free?  May I take a look at it?  Maybe I could merge some of
your ideas into MC, I have busy schedule, but new ideas are always a
good thing.

Best wishes,
Miguel.

and my reply

From dickey Fri Mar 15 10:38:23 1996
Subject: Re: Midnight commander pr
To: miguel@roxanne.nuclecu.unam.mx (Miguel de Icaza)
Date: Fri, 15 Mar 1996 10:38:23 -0500 (EST)
In-Reply-To: <199603151447.IAA12026@roxanne.nuclecu.unam.mx> from "Miguel de Icaza" at Mar 15, 96 08:47:14 am
X-Mailer: ELM [version 2.4 PL24alpha3]
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
Content-Length: 1782
Status: RO

> We had to remove features to make it work under ncurses (for example,
> from the ncurses mailing list, I know that until a couple of weeks
> ago, they still had problems with programs that had their terminal
> resized, ncurses would just crash).  We have had proper terminal
> resizing for a while now.  If ncurses would have been as robust as it
> looks now, we maybe would have stayed with them..
two points there:

        + I reported the resize problem (which is a bug). (It's a long story,
          if you're interested, I'll follow up). (I originally started that
          thread in mid 1994 to _get_ the wresize stuff incorporated).

        + mc 3.1.4 still crashes (even with slang) when resizing rather small.
          (I'm not an mc user -- I've been using it as a test application
          for mods I'm making to xterm to better support color).

> Now, Slang is faster than ncurses (someone posted some benchmarks some
> time ago), so in the long run, we had a faster screen manager.
> Anyways, we can do both ncurses and slang :-).
I saw that.  But I use resizing a lot.  (I once wrote my own screen optimizer -
once is enough).

> > (I have a directory editor which is different in design from MC, which I'm
> > keeping curses-based).
>
> Is it free?  May I take a look at it?  Maybe I could merge some of
> your ideas into MC, I have busy schedule, but new ideas are always a
> good thing.
It's free.  I've not advertised it much for a variety of reasons.  (And it
won't ever look/act like MC, because some of the differences are part of the
design goals).  Basically, I'm holding off on wider stuff til (if) I get my
bugs with ncurses resolved (longer story), and have time to implement
rebindable keys.

You can get a copy at

        ftp.clark.net:/pub/dickey/ded

--
Thomas E. Dickey
dickey@clark.net

helps illustrate the problems.

Reading the Midnight Commander code, the basic problem was not realizing that the curses data structures would have to be adjusted. The difference is that I went ahead and made it work, while on one hand Miguel, et al, decided that slang would solve the problem and on the other, Zeyd dropped the comment about SIGWINCH.

#### ncurses0.5 -> ncurses0.6 #### March 1, 1993 ####
...
* removed SIGWINCH support as SYSV does not define its semantics.

Midnight Commander 3.0 and later (until 2009) bundled a small subset of the slang library, Although that was 5787 lines of code, it was dwarfed by the overall program which had grown to 70629 lines. An (undated) note in 3.2.1's “NEWS” file for 3.0 says

- Slang support, you don't need ncurses anymore (but you can still compile
  with ncurses, if you want).

John Davis, by the way, is one of 18 developers credited in that “NEWS” file.

By comparison, I was a “poor cousin”, but I'm cited in the changelog for August 1997. In 1997, while absorbed in various ncurses issues, I spent some time making a patch for Midnight Commander to make it work properly with ncurses:

From owner-ncurses@mailgate.BSDI.COM  Mon Jun  2 06:15:47 1997
Received: from mailgate.BSDI.COM (mailgate.BSDI.COM [205.230.225.18]) by mail.clark.net (8.8.5/8.6.5) with ESMTP id GAA06453 for <dickey@clark.net>; Mon, 2 Jun 1997 06:15:46 -0400 (EDT)
Received: (from majordomo@localhost) by mailgate.BSDI.COM (8.8.5/8.8.2) id EAA02492 for ncurses-outgoing; Mon, 2 Jun 1997 04:05:33 -0600 (MDT)
Received: from external.BSDI.COM (external.BSDI.COM [205.230.225.1]) by mailgate.BSDI.COM (8.8.5/8.8.2) with ESMTP id EAA02488 for <ncurses@mailgate.bsdi.com>; Mon, 2 Jun 1997 04:05:27 -0600 (MDT)
Received: from mail.clark.net (mail.clark.net [168.143.0.10])
        by external.BSDI.COM (8.8.5/8.8.5) with ESMTP id EAA27539
        for <ncurses@bsdi.com>; Mon, 2 Jun 1997 04:05:24 -0600 (MDT)
Received: from explorer2.clark.net (dickey@explorer2.clark.net [168.143.0.5]) by mail.clark.net (8.8.5/8.6.5) with ESMTP id GAA05313; Mon, 2 Jun 1997 06:04:41 -0400 (EDT)
From: "T.E.Dickey" <dickey@clark.net>
Received: (from dickey@localhost) by explorer2.clark.net (8.8.5/8.7.1) id GAA26253; Mon, 2 Jun 1997 06:05:12 -0400 (EDT)
Message-Id: <199706021005.GAA26253@explorer2.clark.net>
Subject: Re: midnight commander 3.2 and new ncurses under xterm
To: lorenzo@argon.roma2.infn.it (Lorenzo M. Catucci)
Date: Mon, 2 Jun 1997 06:05:11 -0400 (EDT)
Cc: ncurses@bsdi.com (Ncurses Mailing List),
        miguel@roxanne.nuclecu.unam.mx (Midnight Commander),
        mc@roxanne.nuclecu.unam.mx (Midnight Commander List)
In-Reply-To: <Pine.LNX.3.95.970602112319.14348C-100000@argon.roma2.infn.it> from "Lorenzo M. Catucci" at Jun 2, 97 11:26:32 am
X-Mailer: ELM [version 2.4 PL24alpha3]
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
Sender: owner-ncurses@mailgate.BSDI.COM
Precedence: bulk
Status: RO
Content-Length: 1249
Lines: 30

> I'm not sure if it is a gpm, ncurses or xterm problem, but with latest
> ncurses i'm not able to use the mouse inside mc's window any more to
> select or give commands.
both.  Someone reported this week before last, and I spent much of last
week analyzing and devising a workaround for it. I put together a patch
for mc against 3.5.41 (the most recent development version on their site
at that time), and added the first pass of code to make the 4.2 version
incorporate the fix.

The mc patch is available (though you may have trouble resync'ing depending
on the version you have) in

        ftp.clark.net:/pub/dickey/ncurses/4.1/mc-3.5.41-970530-ncurses.patch.gz

(and yes, I sent email late on Thursday to the maintainers of mc, but have not
received a response).

Basically, the fix allows mc to turn off ncurses' mouse support (I chose this
route because mc's style of mouse usage is a little different from that which
ncurses will support - however I also found that the click composition in
ncurses does not work properly, so I added that to my buglist).

> Hope this helps someone find out where is the problem.
thanks anyway - I was unaware of the problem until recently.

--
Thomas E. Dickey
dickey@clark.net
http://www.clark.net/pub/dickey

After a few back-and-forth emails it was integrated into Midnight Commander:

From miguel@athena.nuclecu.unam.mx  Mon Aug  4 15:03:20 1997
Received: from athena.nuclecu.unam.mx (athena.nuclecu.unam.mx [132.248.29.9]) by mail.clark.net (8.8.5/8.6.5) with ESMTP id PAA08628 for <dickey@clark.net>; Mon, 4 Aug 1997 15:03:16 -0400 (EDT)
Received: (from miguel@localhost)
        by athena.nuclecu.unam.mx (8.8.5/8.8.5) id OAA10975;
        Mon, 4 Aug 1997 14:02:46 -0500
Date: Mon, 4 Aug 1997 14:02:46 -0500
Message-Id: <199708041902.OAA10975@athena.nuclecu.unam.mx>
From: Miguel de Icaza <miguel@nuclecu.unam.mx>
To: mc-devel@athena.nuclecu.unam.mx
CC: ado@software-ag.de, dickey@clark.net
Subject: mc 4.0.10 is out.
Status: RO

New version of the developers release of Midnight Commander is out.

this version features:

        - Norbert's patches from today are in: Left/Right -> Up/Down
          changes as suggested on the list; fixes to Slang merger.
          Fixes, fixes, fixes :-)  You will love this version

        - Dickey's patches for ncurses are in.

        - Alexander's Windows and OS/2 updates for the new slang code
          are in

        - rpm -ta mc-4.0.10.tar.gz should work now.

Get this from:

        ftp://ftp.nuclecu.unam.mx/linux/local/devel

Cheers,
Miguel

Here is the entry in OChangeLog from mc-4.1.35 (4.1.35 was released in May 1998, 4.0 was released in June 1997):

Mon Aug  4 18:27:43 1997  T.E. Dickey <dickey@clark.net>

        * change declaration of keyok() to 'int' rather than 'void'.

        * configure.in: add test for 'resizeterm()' and 'keyok()' (I did
          test builds against ncurses 4.1, 4.0, and 1.9.9e).

          Btw, I tested resizing on SunOS 4.1, with both ncurses and slang.
          Your application handles a few resizing events before getting hung
          (this is not related to the libraries - perhaps you should
          investigate it further).

        * Makefile.in: remove a couple of items from the distclean rule
          that don't correspond to generated files.

        * resize (supported in NCURSES 4.0)

        * mouse (supported in NCURSES since 1.9.6, but not compatible with MC
          -- the simplest solution is to turn off the mouse code in NCURSES so
          that MC can do what it wants.  I've constructed a patch that will do
          this - the function will be in the next release of NCURSES; it was a
          minor item on my list, so I did it this week to get MC working).

        * some configure nits (if ncurses is installed as curses, your script
          does the wrong thing).

        * vt100-style codes for F1-F4 (you'll need these for the XFree86 xterm)

        * add some stuff to the "make distclean" rule, so it _is_ clean.
          (Otherwise I waste time making the patch).

        * initialized a variable in menu.c (your program dumped core while I
          was testing the mouse events).

Keeping that in perspective, the file is 12744 lines. My contribution was 33 lines to that change-log, and I was not mentioned in the accompanying FAQ as a contributor. The FAQ, by the way, listed several lesser contributors, and overlooked some who did far more work, such as Pavel Roskin (i.e., only about one third of the contributors were listed in the FAQ).

After that, I occasionally reported a problem or sent a patch. Referring to the git log:

commit d3887737791283e676823ba84dca173be07395b4
Author: Pavel Roskin <mail-address>
Date:   Wed Jul 23 05:44:25 2003 +0000

    * key.c (xterm_key_defines): Add more sequences for xterm-179 by
    Thomas E. Dickey.

commit 5257da5f83b80074b93d66fcaca7672afd6a07f5
Author: Pavel Roskin <mail-address>
Date:   Wed Feb 6 05:23:50 2002 +0000

    * key.c [USE_NCURSES] (get_key_code): Ignore KEY_RESIZE.
    * layout.c [USE_NCURSES] (flag_winch): Don't call
    low_level_change_screen_size() from signal handler.
    From Thomas Dickey <mail-address>

commit 9a67ad6ebfbbd93a9d3fc885b58c9510fa2fefc9
Author: Pavel Roskin <mail-address>
Date:   Wed Feb 6 05:14:16 2002 +0000

    * screen.c (format_file): Add a new parameter (size of the
    output buffer) to prevent buffer overflow on very wide
    terminals.
    From Thomas Dickey <mail-address>

commit e9eac440ff1717c66636d646f019c66794de790d
Author: Pavel Roskin <mail-address>
Date:   Wed Feb 6 05:06:16 2002 +0000

    * configure.in: Don't use defunct ncurses_version.
    Reported by Thomas Dickey <mail-address>
    Set force_ncurses whenever --with-ncurses is used.  If set,
    suppress checks for S-Lang and fail is ncurses is not found.

commit 727898de198aa9d3d8884d80377ea4ed4c394a47
Author: Pavel Roskin <mail-address>
Date:   Wed Feb 6 04:38:38 2002 +0000

    * configure.in: Introduce a new variable screen_type that is
    either "slang" or "ncurses" without additional text.  Use it
    instead of screen_manager whenever possible.  Fix text in
    AC_NCURSES call, use m4 quotes and move shell quotes into the
    macro.
    * acinclude.m4: Likewise.
    Reported by Thomas Dickey <mail-address>

I also reported this (after subscribing to the mailing list...), but was not mentioned:

commit 5978bcc19612edd479cebd1dcee07faa753aa691
Author: Pavel Tsekov <mail-address>
Date:   Mon Sep 18 21:06:58 2006 +0000

    * src/key.c (get_key_code): Do not blindly strip the eight bit of the
    key code when the 'use_8th_bit_as_meta' flag is set. Only key codes
    in the range 128 - 255 should be stripped, the rest must be left
    intact.

Reflecting on that reminds me of an episode a couple of years later:

On the positive side, shortly afterwards, the bundled subset of slang was dropped. Many people confused that with the full-sized library, and used that in comments that slang was smaller, lighter, etc. Smaller is quantifiable (it is not smaller), and “lighter” is not.

In researching this section, I collected copies of the few remaining early tar-balls. There is not much to report:

The one promising ftp site is dead.
There is a git repository, with the change-log cleverly directing the reader to the git log.
The git log only goes back to January 1, 1997,
but the first few commits were corrupt and actual history begins in February 1998
(for instance, John Davis is mentioned once, while I am mentioned 6 times).
The first 4 years of development history appear to be gone forever,
except for people who collect tar-balls.

Symbols used by tie-ins

These versions were used to obtain the lists of symbols:

lynx – 2.8.9dev.16
mutt – 1.8.3
mc – 4.8.19

In addition to functions, each program uses macros and data definitions. To keep the comparison simple, only symbols shown by nm are listed.

curses/ncurses symbols

Function	lynx	mutt	mc	Notes
COLORS			yes	SVr4, X/Open
COLS			yes	SVr4, X/Open
ESCDELAY			yes	ncurses, adapted from AIX
LINES			yes	SVr4, X/Open
acs_map			yes	SVr4 (not in X/Open)
assume_default_colors	yes			ncurses (1999/11/13)
baudrate			yes	SVr4, X/Open
beep		yes	yes	SVr4, X/Open
cbreak	yes	yes	yes	SVr4, X/Open
clearok	yes	yes	yes	SVr4, X/Open
curs_set		yes		SVr4, X/Open
curses_version	yes	yes		ncurses (1999/10/23)
def_prog_mode	yes			SVr4, X/Open
define_key	yes			ncurses (1997/05/31)
delscreen	yes			SVr4, X/Open
delwin	yes			SVr4, X/Open
derwin	yes			SVr4, X/Open
doupdate	yes		yes	SVr4, X/Open
echo	yes			SVr4, X/Open
endwin	yes	yes	yes	SVr4, X/Open
flushinp		yes	yes	SVr4, X/Open
getmouse	yes			ncurses (1995/09/17)
has_colors	yes	yes	yes	SVr4, X/Open
init_pair	yes	yes	yes	SVr4, X/Open
initscr		yes	yes	SVr4, X/Open
is_wintouched			yes	SVr4, X/Open
isendwin		yes		SVr4, X/Open
key_defined		yes		ncurses (2003/03/08)
keypad	yes	yes	yes	SVr4, X/Open
meta		yes	yes	SVr4, X/Open
mouseinterval	yes			ncurses (1995/10/17)
mousemask	yes			ncurses (1995/10/17)
napms	yes	yes		SVr4, X/Open
newpad	yes			SVr4, X/Open
newterm	yes			SVr4, X/Open
newwin	yes			SVr4, X/Open
nl	yes			SVr4, X/Open
nocbreak			yes	SVr4, X/Open
nodelay			yes	SVr4, X/Open
noecho	yes	yes	yes	SVr4, X/Open
nonl	yes			SVr4, X/Open
noraw			yes	SVr4, X/Open
pair_content	yes			SVr4, X/Open
pnoutrefresh	yes			SVr4, X/Open
printw		yes		SVr4, X/Open
raw			yes	SVr4, X/Open
reset_prog_mode			yes	SVr4, X/Open
reset_shell_mode			yes	SVr4, X/Open
resizeterm	yes	yes	yes	ncurses (1996/09/17)
scrollok	yes	yes		SVr4, X/Open
start_color	yes	yes	yes	SVr4, X/Open
stdscr			yes	SVr4, X/Open
tgetstr			yes	SVr4, X/Open
tigetflag		yes		SVr4, X/Open
tigetstr	yes	yes		SVr4, X/Open
typeahead		yes		SVr4, X/Open
ungetmouse	yes			ncurses (1995/09/17)
use_default_colors	yes	yes	yes	ncurses (1997/01/18)
use_extended_names		yes		ncurses (1999/03/01)
use_legacy_coding	yes			ncurses (2005/12/17)
vwprintw		yes		SVr4, X/Open (deprecated)
wadd_wch		yes		X/Open
waddch	yes	yes	yes	SVr4, X/Open
waddnstr	yes	yes	yes	SVr4, X/Open
wattr_get	yes			X/Open
wattr_off	yes			X/Open
wattr_on	yes			X/Open
wbkgd	yes			SVr4, X/Open
wbkgdset	yes	yes		SVr4, X/Open
wborder	yes			SVr4, X/Open
wclear	yes	yes		SVr4, X/Open
wclrtobot		yes		SVr4, X/Open
wclrtoeol	yes	yes		SVr4, X/Open
werase	yes			SVr4, X/Open
wgetch	yes	yes	yes	SVr4, X/Open
whline			yes	SVr4, X/Open
wmove	yes	yes	yes	SVr4, X/Open
wnoutrefresh	yes			SVr4, X/Open
wrefresh	yes	yes	yes	SVr4, X/Open
wresize	yes			ncurses (1996/09/07)
wtimeout		yes		SVr4, X/Open
wtouchln	yes		yes	SVr4, X/Open
wvline			yes	SVr4, X/Open
Total: 83	51	37	38

Notes:

With the exception of the wide-character function wadd_wch, ncurses support for the X/Open functions listed here was completed during 1995.
Wide-character support was added in several stages, e.g., wadd_wch beginning in 2001.
As a special case, wattr_on, etc., were macros that were later revised to take into account the wide-character changes.
Some “SVr4” features originated in much older versions of curses. I cited SVr4 here because that corresponds closely to Solaris curses during the early/mid-1990s.
The mouse-functions predate my detailed change-list, first seen in September 1995.
Midnight Commander uses the termcap interface for compatibility with slang, which (even in August 2017) uses termcap names. Oddly enough, the program tries twice (first using terminfo names, then using the two-character termcap names), but the first try fails (see mailing list and source-code).
A 20-year-old note in Midnight Commander's HACKING explains the generally lower-level functions and global variables (rather than functions) used for both ncurses and slang (compare with Lynx):
```
Although S-Lang is now used by default, we still support ncurses.  We
basically are using a small subset of ncurses because we want to be
compatible with Slang.
```

slang symbols

Function	lynx	mutt	mc	Notes
SLang_TT_Baud_Rate			yes	1994/11/17
SLang_TT_Read_FD			yes	1995/09/18
SLang_Version			yes	1993/11/16 undocumented
SLang_create_keymap	yes			1994/11/17
SLang_do_key	yes			1994/11/17
SLang_exit_error	yes			1994/11/17
SLang_flush_input	yes			1994/11/17
SLang_get_error	yes			2004/10/11
SLang_getkey	yes		yes	1994/11/17
SLang_init_tty	yes		yes	1994/11/17
SLang_input_pending	yes		yes	1994/11/17
SLang_reset_tty	yes		yes	1994/11/17 undocumented
SLang_undefine_key	yes			1994/11/17
SLcurses_cbreak		yes		1996/09/03 undocumented
SLcurses_clearok		yes		1996/11/20 undocumented
SLcurses_delwin		yes		1996/09/03 undocumented
SLcurses_endwin		yes		1996/11/20 undocumented
SLcurses_getch		yes		1996/09/03 undocumented
SLcurses_has_colors		yes		1996/09/03 undocumented
SLcurses_initscr		yes		1996/09/03 undocumented
SLcurses_newwin		yes		1996/09/03 undocumented
SLcurses_nil		yes		1996/09/03 undocumented
SLcurses_printw		yes		1996/11/20 undocumented
SLcurses_start_color		yes		1996/09/03 undocumented
SLcurses_waddch		yes		1996/09/03 undocumented
SLcurses_waddnstr		yes		1996/11/20 undocumented
SLcurses_wattrset		yes		1996/11/20 undocumented
SLcurses_wclear		yes		1996/11/20 undocumented
SLcurses_wclrtobot		yes		1996/09/03 undocumented
SLcurses_wclrtoeol		yes		1996/09/03 undocumented
SLcurses_wmove		yes		1996/09/03 undocumented
SLcurses_wrefresh		yes		1996/09/03 undocumented
SLexpand_escaped_string	yes			1994/04/12 undocumented
SLkm_define_keysym	yes			1996/11/12
SLsignal			yes	1996/11/20 undocumented
SLsmg_Display_Eight_Bit			yes	1995/02/23
SLsmg_draw_box	yes			1994/11/17
SLsmg_draw_hline			yes	1994/11/17
SLsmg_draw_object			yes	1994/11/17
SLsmg_draw_vline			yes	1994/11/17
SLsmg_erase_eol	yes			1994/11/17
SLsmg_erase_eos	yes			1994/11/17
SLsmg_fill_region	yes		yes	1994/11/17
SLsmg_forward	yes			1996/09/03
SLsmg_get_column	yes		yes	1995/09/18
SLsmg_get_row	yes		yes	1995/09/18
SLsmg_gotorc	yes		yes	1994/11/17
SLsmg_init_smg	yes	yes	yes	1994/11/17
SLsmg_normal_video			yes	1994/11/17
SLsmg_refresh	yes		yes	1994/11/17
SLsmg_reinit_smg			yes	1999/06/06
SLsmg_reset_smg	yes	yes	yes	1994/11/17
SLsmg_resume_smg	yes		yes	1995/09/18
SLsmg_set_char_set			yes	1994/11/17
SLsmg_set_color	yes		yes	1994/11/17
SLsmg_suspend_smg	yes			1995/09/18
SLsmg_touch_lines	yes		yes	1994/11/17
SLsmg_touch_screen	yes			1999/05/17 undocumented
SLsmg_vprintf			yes	1994/11/17
SLsmg_write_char	yes		yes	1994/11/17
SLsmg_write_nstring	yes			1995/09/18
SLsmg_write_string			yes	1994/11/17
SLtt_Blink_Mode			yes	1994/11/17 undocumented
SLtt_Has_Alt_Charset			yes	1995/09/18 undocumented
SLtt_Ignore_Beep			yes	1994/11/17 undocumented
SLtt_Screen_Cols			yes	1994/11/17
SLtt_Screen_Rows			yes	1994/11/17
SLtt_Try_Termcap			yes	1995/09/18 undocumented
SLtt_Use_Ansi_Colors			yes	1994/11/17
SLtt_add_color_attribute	yes		yes	1995/09/18 undocumented
SLtt_beep		yes	yes	1994/11/17 undocumented
SLtt_flush_output	yes			1994/11/17 undocumented
SLtt_get_screen_size	yes		yes	1996/09/03 undocumented
SLtt_get_terminfo	yes		yes	1994/11/17
SLtt_set_color	yes	yes	yes	1994/11/17
SLtt_set_color_object		yes		1994/11/17 undocumented
SLtt_set_cursor_visibility		yes		1995/09/18 undocumented
SLtt_set_mono	yes		yes	1994/11/17 undocumented
SLtt_set_mouse_mode	yes			1995/09/18 undocumented
SLtt_tgetnum			yes	1995/09/18 undocumented
SLtt_tgetstr			yes	1995/09/18 undocumented
SLtt_write_string			yes	1994/11/17 undocumented
SLtty_set_suspend_state	yes			1995/09/18
SLutf8_enable		yes	yes	2004/10/11
SLvsnprintf		yes		1998/04/28 undocumented
Total: 85	38	27	45

Notes:

Because there are no dates in slang's change-logs, the versions and dates cited use the tar-balls, etc., which were available at the time of writing this page.
Initially, none of the slang symbols were documented. I have not presented a history showing when the documentation was provided, only dates when the symbol was first provided in the source code. Those indicated as undocumented are not described in the slang documentation as of August 2017. A mention in “undoc.tm” does not count as documentation.
A few of the undocumented features were documented in earlier versions, e.g., SLang_Version, but the SLcurses interface was never documented beyond a couple of lines of text.

John Davis has his own opinion on the topic of SLcurses (emphasis added):

From davis@space.mit.edu Thu Apr 24 11:28:58 2003
Path: news1.radix.net!newspeer.radix.net!news-spur1.maxwell.syr.edu!news.maxwell.syr.edu!newsfeed.icl.net!newsfeed.fjserv.net!logbridge.uoregon.edu!newsfeed.stanford.edu!bloom-beacon.mit.edu!senator-bedfellow.mit.edu!dreaderd!not-for-mail
From: davis@space.mit.edu (John E. Davis)
Newsgroups: alt.lang.s-lang
Subject: Re: S-Lang C library
References: <slrnbaft44.sfg.anzuhan@ihme.ihme.org>
Organization: Center for Space Research
Reply-To: davis@space.mit.edu
Message-ID: <slrnbafurr.50s.davis@aluche.mit.edu>
User-Agent: slrn/0.9.6.4 (Linux)
Date: 24 Apr 2003 14:57:27 GMT
NNTP-Posting-Host: ALUCHE.MIT.EDU
X-Trace: 1051196247 senator-bedfellow.mit.edu 3944 18.75.2.45
Xref: news1.radix.net alt.lang.s-lang:21660
Status: RO
Content-Length: 933
Lines: 22

On Thu, 24 Apr 2003 14:28:15 GMT, Antoine Kalmbach <anzuhan@ihme.org>
wrote:
>I have been using the ncurses libary for a while now and I also
>tried the S-Lang library. Even though the ncurses is a bit more
>easier than S-Lang, I like S-Lang. Are there any tutorials out
>there or do I have to read sources i.e. of mutt/slrn?

Mutt uses slang's curses emulation layer, which I do not recommend
using.  The emulation was written primarily for me to test the SLsmg
routines by taking advantage of several test programs written for
curses.  As such, slang's curses emulation is incomplete.

Instead, I recommend using slang's native SLsmg functions.  Programs
that use these functions include jed, slrn, and I believe the newt
library.

As far as I know, there are no tutorials.  I believe the slang SLsmg
routines were featured in a chapter of some "programming under linux"
book, but I do not remember the details.

Good luck.
--John

General discussion

Library name

Regarding the library name, I have always treated it as a pun on curses, considering that the main use of the library is for screen-management.

According to John Davis, that is not how the name came about. In his reference manual, he says

S-Lang was originally a stack language that supported a postscript-like syntax. For that reason, I named it S-Lang, where the S was supposed to emphasize its stack-based nature. About a year later, I began to work on a preparser that would allow one unfamiliar with stack based languages to make use of a more traditional infix syntax. Currently, the syntax of the language resembles C, nevertheless some postscript-like features still remain, e.g., the `%' character is still used as a comment delimiter.

But that is only part of the story. While the foldoc description gives the earliest (unverifiable) date for a version of S-Lang, it also mentions postfix (not synonymous with PostScript) and Forth. The earliest verifiable source I have found (0.95-2, November 1993) contains none of those names. In its slang.txt, one may read

S-Lang (pronounced ``sssslang'') is a powerful stack based language
interpreter with a C-like syntax which may be easily embedded into another
application, making it extensible.
...
The syntax of the language is quite simple and is very similar to C.  Unlike
C, S-Lang variables are untyped and inherit a type upon assignment. The
actual type checking is performed at run time.  In addition, there is
limited support for pointers.
...
   4. Comments are started with the `%' character and extend to the end of the
      line.  In C, the `%' character denotes the mod operation. S-Lang
      denotes the mod operation by `mod'.

Given the foldoc summary, and the episodic nature of his releases, Davis could have started this library in April/May 1992, assuming that the statement “About a year later” is accurate. But he says “sometime in the fall of 1992”, which is at variance from the foldoc comment.

The word “forth” appears in 0.99-11, but it is a misspelling for “fourth” (and is carried through to the present). To save the reader suspense on this point:

The terms “PostScript” and “postfix” in the sense implied in the foldoc quote first appeared in slang's documentation early in 1998 when Davis started to document the library. But that is what Davis has on his website.
In a make-file, he had an earlier use of "postfix" as an alternative for "suffix", which need not concern the reader. When referring to “PostScript”, Davis always lowercases the word, apparently viewing it as a generic term.

However, Davis did not start by naming the language S-Lang. I pointed out a problem with the library name in 1999, to which he responded:

From davis@space.mit.edu Tue Jun 29 12:38:41 1999
Received: from smtp-gw.vma.verio.net (smtp-gw.vma.verio.net [207.97.20.30])
        by ice.clark.net (8.8.8/8.8.8) with ESMTP id MAA08928
        for <dickey@clark.net>; Tue, 29 Jun 1999 12:38:41 -0400 (EDT)
Received: from space.mit.edu (SPACE.MIT.EDU [18.75.0.10])
        by smtp-gw.vma.verio.net (8.9.3/8.9.3) with SMTP id MAA07293
        for <dickey@clark.net>; Tue, 29 Jun 1999 12:38:50 -0400 (EDT)
Received: from aluche.mit.edu by space.mit.edu AA11300; Tue, 29 Jun 99 12:38:31 EDT
Received: (from davis@localhost)
        by aluche.mit.edu (8.8.8/8.8.7) id MAA02430;
        Tue, 29 Jun 1999 12:38:30 -0400
Date: Tue, 29 Jun 1999 12:38:30 -0400
From: "John E. Davis" <davis@space.mit.edu>
Message-Id: <199906291638.MAA02430@aluche.mit.edu>
To: dickey@clark.net
Subject: Re: slang (QNX)

Status: RO
Content-Length: 390
Lines: 10

On Tue, 29 Jun 1999 09:39:18 -0400 (EDT), you said:
>Are you aware that someone's selling something which looks like (and is named
>the same) as your slang library?

No I was not.  Fortunately, it is not my library tha they are selling.
I believe that there is already a language called `Slang', which is
the reason I had to re-name mine (very early on) `S-Lang'.

Thanks,
--John

and also this clarification:

From dickey Tue Jun 29 13:18:59 1999
Subject: Re: slang (QNX)
To: davis@space.mit.edu (John E. Davis" <davis@space.mit.edu>)
Date: Tue, 29 Jun 1999 13:18:59 -0400 (EDT)
In-Reply-To: <199906291701.NAA02505@aluche.mit.edu>  from "John E. Davis" <davis@space.mit.edu>" at Jun 29, 99 01:01:39 pm
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text
Status: RO
Content-Length: 342
Lines: 16

>
> On Tue, 29 Jun 1999 12:59:22 -0400 (EDT), you said:
> >to my uneducated eyes, the typeless-C language looks like S-Lang.
> >(I'd keep an eye on them).
>
> Thanks, I will also.  They claim theirs is based on lisp.

it could be awkward if it is not.

> --John

I probably saw "Specware 2.0.3 (March 1998)", documented here (at Michigan State University).

Davis' comment about a pre-existing language could have been this:

SLANG a problem solving language for continuous-model simulation and optimization (doi 10.1145/800195.805913). Joe M. Thames, Jr., Proceeding ACM '69 Proceedings of the 1969 24th national conference. Pages 23-41. August 26-28, 1969.

SLANG is a mathematical problem modeling and solution language. It is one of several languages in the programming subsystem of the Computer User Executive (CUE) System developed by TRW Systems. SLANG is both a procedural and a command language designed primarily for the “casual” user. Consequently, much attention was paid to programming ease, “natural” syntax rules, readability, and debugging ease. On the other hand, SLANG is designed to permit the solution of very sophisticated mathematical problems, characterized by iterative solution methods. Its translated object code therefore contains complex numerical solution logic in addition to the object code of its procedural syntax.
SLANG(ID:470/sla008)

Digital simulation language from TRW, charactersed by naturalness and ease of use.

Those are the same: Thames' paper says he was with the TRW Systems Group in California. Thames' slang is procedural, influenced by Algol, unlike C (i.e., it uses keywords rather than punctuation for loops).

Rather than keywords, slang uses punctuation, like C, although some of the choices of punctuation differ. Here is a fragment of the calc.sl demo from slang 0.95-2 to illustrate:

% Newton's method requires the derivative of a function.  Here is such a
% function called by newton.  Given f(x), it returns df/dx at the point x.
%
% Its usage is:
%
%    derivative(x, &f);
define derivative(x, f)
{
   variable dx;
   dx = 1.0e-4;        % small number

   return ((f(x + dx) - f(x - dx))/(2 * dx));
}

% And now the Newton's method:

define newton(x, f)
{
   variable err, max, dx;

   err = 1.0e-6;
   max = 1000;

   while (max)
     {
        --max;
        dx = f(x) / derivative(x, &f);
        if (abs(dx) < err)
          {
             return(x);
          }

        x -= dx;   % equivalent to: x = x - dx;
     }

   print ("\7Root not found.  Try another seed!\n");
   return(x);
}

Contrast that with Joseph H. Allen's ivy, which was posted a few months earlier to alt.sources . Here is the bubble-sort example from that posting:

# Bubble sort function

:sort array size
  loop                                  # Loop until sorted
    flg=0
    for a=0, a!=size-1, ++a             # Check array
      if array[a]>array[a+1]            # Element out of order?
        array[a]=array[a+1]:array[a]    # Swap it...
        flg=1
    until !flg
  return array

# Example uses

print sort[{3 8 2 6 4 5 1 9} 8]
print sort[{"this" "is" "a" "test" "of" "this" "thing"} 7]

Back to the point: as used in Lynx, Mutt, and Midnight Commander, it is an alternative to ncurses which has acquired the connotation of a pun. If that was never Davis' intention, it is merely an ironic coincidence.

Screen management

The first mention of screen management came in 0.99-7 (November 1994), about a year after I first examined ncurses (e.g., 1.8.1). In 0.99-7, Davis used Linux-specific code in screen management (in sldisplay.c):

void SLtt_beep (void)
{
   if (SLtt_Ignore_Beep & 0x1) SLtt_putchar('\007');

   if (SLtt_Ignore_Beep & 0x2)
     {
        if (Visible_Bell_Str != NULL) SLtt_write_string (Visible_Bell_Str);
#ifdef linux
        else if (Linux_Console)
          {
             SLtt_write_string ("\e[?5h");
             SLtt_flush_output ();
             usleep (50000);
             SLtt_write_string ("\e[?5l");
          }
#endif
     }
   SLtt_flush_output ();
}

There was no screen-management in the previous snapshot 0.99-2 (May 1994), hinting that he began development of the screen management around the time he started developing with Linux (note that Davis began using Linux late in 1993).

The (undated) entries in the change-log are of no use in getting more precise dates. Nor are the copyright dates useful. In 0.95-2 one source file was marked 1992 and 1993, but the other 14 files were marked 1993. In the following year (seen in 0.99-2) all of those files were modified to say 1992 and 1994. Later, in 0.99-7, Davis

added 9 files which (initially) were not back-dated to 1992: sldostty.c, slerr.c, slgetkey.c, slkeymap.c, slos2tty.c, slrline.c, sltoken.c, slutty.c, slvmstty.c, and
added 10 files with 1992 and 1994 dates: slasync.c, sldisply.c, slfile.h, slmatrix.c, slmemchr.c, slmemcmp.c, slmemcpy.c, slmemset.c, slvideo.c, slxstrng.c.

The earlier releases were distributed through DECUS (Digital Equipment Computer Users' Society), and provided an Alpha executable for the calculator demo in the first release. Although there were makefiles for DJGPP, MS-DOS, and OS/2 in those releases, the November 1994 version was the first to introduce a variety of system-dependent features. In the source-code, Davis uses the acronym smg, influenced by the VMS SMG$ facility. Having first seen it in late 1984 (too late for me to be interested with flist), I know that it had been around for about ten years at that point.

At this point in time (1994), Davis' experience with curses was apparently based on VMS curses and SunOS 4. For the sake of example, none of these supported color:

VMS header file) from 1995,
4.4BSD-Lite2 header file) from 1995,
SunOS 4 header file) from 1994, and (almost identical)
4.2BSD header file) from 1983.

The A_xxx names in the VMS header file may have come from SVr2 (1984), or the absence of A_COLOR may reflect the lack of color support by DEC's terminals (according to what I read in the mid-1990s, the VT525 — released in 1994 — was a clone of the Wyse-370, although DEC's termcaps did not mention color). Whether that was accurate or not, the low-end models did not support ANSI color, according to Bill Hedberg's response to wyse150, vt220/vt240 color problems:

In article <3v3vqp$q...@maverick.tad.eds.com>, dz2...@csd.can.eds.com (Rag Ramanathan) writes...

> Can anyone familiar with vt220 (mono) and vt240 (rgb only ?? )
> escape sequnces help me with the followig problem(s):
>
> I am trying to use setf, setb capabilities which are supposed to set
> the foreground and background colors.
>
> Is setf=^[[3%p1%dm  and setb=^[[4%p1%dm for vt220 and vt240
>
> To test the above I used:
>
>         echo "\033[30m \033[47m Hello"
>         echo "\033[37m \033[47m Hello"
>
> These seem to have no effect on either vt220 or vt240. But they work as
> expected on wyse 370(color) and color aixterm.

 The above control sequences are part of the ANSI SGR - Set Graphics Rendition
 command and were not implemented in the VT220 or VT240.  They are implemented
 the the VT525 color text terminal.

> Why does not the programmers pocket guide for vt220/vt240 say anything
> about color. What are the effects of the above escape sequence on a
> mono terminal - should'nt it atleast do reverse video as in tput rev.

> Why does tput bold on vt240 change the text foreground color to red while
> the normal text is green on black.  The same terminal shows blue
> background and black foreground for tput rev.  Is there some thing
> wrong with the setup.

 Character attributes such as bold are mapped to color in the VT240,
 actually VT241 which is the same system box, but connected to a color
 monitor.  There should be a SetUp screen which allows the user to
 specify colors.

To get color, a developer needed something more recent than 1984, e.g., SVr3 (1986) or SVr4 (1988).

Just for the record,

I used an earlier version of VMS curses on a project in 1989, and found that the feature for switching terminal modes for getch did not work properly. I used the debugger to disassemble the library module containing getch to understand the problem.
But I did not (much) use SunOS 4 until 1994, since I used an Apollo workstation in the 1987-1994 timeframe. Initially those used the Aegis operating system, but later Apollo introduced environments for 4.3BSD and SVr3 applications (actually Apollo's marketing people said those were “co-resident kernels” rather than environments).

A few others in my organization had Sun3's and then Sun4's, but the majority used Apollo's. I found both (kernels or environments) useful for developing ded. By the time I became involved with Lynx in 1997, VMS curses was usable, and SVr4 curses on Unix systems was taken for granted.

The sldisply.c (S-Lang display) module had a function SLtt_get_terminfo. Actually, it used the termcap interface (with too small a buffer), but modified the strings returned from tgetstr to remove either termcap- or terminfo-padding:

        /* lose pad info --- with today's technology, term is a loser if
           it is really needed */
        while ((*what == '.') ||
               ((*what >= '0') && (*what <= '9'))) what++;
        if (*what == '*') what++;

        /* lose terminfo padding--- looks like $<...> */
        w = what;
        while (*w) if ((*w++ == '$') && (*w == '<'))
          {
             w1 = w - 1;
             while (*w && (*w != '>')) w++;
             if (*w == 0) break;
             w++;
             wsave = w1;
             while ((*w1++ = *w++) != 0);
             w = wsave;
          }
        if (*what == 0) what = NULL;

Although he stripped out padding and delays,

the tt_write function provided crude flow control (by sleeping for a second after writing each second's worth of data if the baud-rate was in the range 300 to 9600) and
the Linux-specific feature would normally (in either termcap or terminfo) be implemented using delays, e.g.,
```
flash=\E[?5h$<200/>\E[?5l,
```

Both of those quirks remain in the library as of August 2017. Apparently the former has been cited as a feature, since someone commented on it:

Posted: Sat Mar 24, 2012 4:27 pm

Apparently they both have text mode features that programs need.

I suppose it's hard to tell which is "better" as they're different. Ncurses has its roots in curses which is a classic UN*X library so it sticks around. Slang is somewhat of a newcomer, it's apparently a full interpreted programming language that has text windowing features, and these windowing features is what programs use more. It also seems to claim to have text spew reduction, which speeds up usage on slow speed terminals (which of course tends not to be an issue anymore...)

Ncurses is smaller as it doesn't have a full interpreter...

I suppose we just have to use whichever library our favorite apps use. Unless you're willing to rewrite for the other library? I hate slang. Very confusing to websearch...

So much for padding not needed for terminals. As for the latter, Davis later modified the check in 0.99-21 to also check for “linux” though his notes are not specific:

Changes since 0.99.20
...
3.  The terminal type `linux' is now recognized by the SLtt interface---
    even if there is no termcap/terminfo entry for it.

and still later in 0.99-38, changed the default setting of SLtt_Ignore_Beep to hide the feature.

The system-specific “linux” supplanted the generic “console” name sometime during 1995, as I pointed out here.

However, Davis compensated for that by adding a comment just before hard-coding the line-drawing feature, in 0.99-21:

   /* I have had it with defective console/linux terminfo entries provided
    * by ncurses.  They are not reliable.  I cannot track every ncurses
    * release and fudge my stuff to work with defective stuff.  For that
    * reason, I am hardcoding some things.
    */
   if (Linux_Console)

Davis' settings did not match those used in ncurses. Although he removed those settings in 0.99-32, his comment stayed in the source until 1998.

In 0.99-7, ANSI color was supported, but hard-coded. Line-drawing was not supported, although the capabilities were read from termcap.

In 0.99-11 (February 1995) Davis added sltermin.c, and other files again back-dated to 1992:

the module contains SLtt_tigetent (essentially a replacement for tgetent which read compiled terminfo rather than termcap text).
most of the comments in the source were copied from the SGI term(4) manual page,
this read a particular flavor of binary-terminfo (see Terminfo format),
it used hand-built tables to map termcap capability names to the offsets in the string-table,
the table was incomplete (60 items versus 298 distinct capabilities in 4.4BSD termcap, or 497 capabilities defined for terminfo), listing only features used in slang,
there was no support for terminfo capability names, and
color was still hard-coded.

Also in 0.99-11 (not mentioned in changes.txt):

untic, printed the termcap information that slang loaded from a compiled terminfo file. All of the capabilities that were tabulated in slang were printed.
worm, attributed to Eric P. Scott at Caltech in 1980. This was provided as a demo in ncurses 1.8.1, but in 0.99-11 Davis modified it to use color and to draw a box..

That brings up a point: the initial slang screen management functions did not make effective use of optimization. In the worm demo, Davis chose to redraw the box each time the screen was updated, just in case the worm trail crossed the box boundaries.

Here is a screenshot:

The curses library does more than one type of optimization:

reducing the number of characters sent to the screen when changing the display (incidentally moving the cursor), and
moving the cursor itself efficiently around the screen.

S-Lang does both of those:

SLsmg_refresh, added in 0.99-7 as part of the first screen management functions, acts much like the curses refresh function. Unsurprisingly enough, the initial slcurses.h file defined “refresh” to that function.
SLsmg_gotorc and SLtt_goto_rc, also added in 0.99-7 correspond roughly to the curses move and mvcur functions, respectively. The first moves the cursor in the virtual screen which will be refreshed later, while the second moves the cursor immediately on the physical screen.

However, none of those slang functions is designed for multiple windows. They assume that there is just one window which uses the entire screen. In curses, that worm demo could use two windows (one for the worm and one for the box) and “touch” the box before refreshing to avoid redrawing it. Better still, one could use the panel library, and not touch the box at all.

Examining the S-Lang screen management functions shows some changes over the years, but some of those changes were of questionable value.

For instance, in 1.2.2 this chunk appeared, and has lasted to the present (late 2017):

/* It appears that the Linux console, and most likely others do not
* like scrolling regions that consist of one line.  So I have to
* resort to this stupidity to make up for that stupidity.
*/
static void delete_line_in_scroll_region (void)
{
   SLtt_goto_rc (Cursor_r - Scroll_r1, 0);
   SLtt_del_eol ();
}

Actually the Linux console was correct. A VT100 scrolling region requires two or more lines. The interested reader may investigate the documented description of DECSTBM (DEC set top/bottom margins) at vt100.net. In this case, and some others, Davis apparently had in mind ansi.sys.

To recap the evolution of slang's screen management features, it began simply enough, but over time (and with competition from ncurses) added features and quirks.

Some pundits read the summary of screen management in S-Lang's documentation:

8. Screen Management

The S-Lang library provides two interfaces to terminal independent routines for manipulating the display on a terminal. The highest level interface, known as the SLsmg interface is discussed in this section. It provides high level screen management functions for manipulating the display in an optimal manner and is similar in spirit to the curses library. The lowest level interface, or the SLtt interface, is used by the SLsmg routines to actually perform the task of writing to the display. This interface is discussed in another section. Like the keyboard routines, the SLsmg routines are platform independent and work the same on MSDOS, OS/2, Unix, and VMS.

and misunderstand the term “high level” and suppose it means more than it does. On occasion I have responded to those comments, as in this newt vs. cdk vs. ?? thread on the comp.os.linux.development.apps newsgroup:

From dickey Fri May 14 09:07:20 1999
From: "T.E.Dickey" <dickey@shell.clark.net>
Subject: Re: newt vs. cdk vs. ??
Newsgroups: comp.os.linux.development.apps
References: <slrn7jjbl5.t1r.grante@isis.visi.com>  <slrn7jkq90.eis.cbbrowne@godel.brownes.org> <slrn7jmgln.4p2.grante@isis.visi.com> <kmK_2.303$6f.5611@news2.giganews.com>
Organization: Clark Internet Services, Inc., Ellicott City, MD USA
User-Agent: tin/pre-1.4-19990413 ("Endemoniada") (UNIX) (SunOS/5.6 (sun4u))
Status: RO
Content-Length: 400
Lines: 12

Christopher Browne <cbbrowne@news.hex.net> wrote:

> Fair enough.  SLANG has the valuable features that:
> a) It seems to provide *some* higher-level abstractions than does
> curses, and

aside from the language interpreter, no one's actually cited any examples
of higher-level abstractions that slang does which curses does not.

--
Thomas E. Dickey
dickey@clark.net
http://www.clark.net/pub/dickey

but as you may read, that has little effect. I also recall (seriously) someone arguing that slang was “higher level” than ncurses because its functions were named more consistently, e.g., SLsmg_xxx, etc.

Library size

Aside from the programs which John Davis writes, almost all of the applications using the library use only the curses-related functions. I pointed out in the manual for dialog early in 2007 that the slang library was larger than ncurses, that (even with the additional features of dialog) dialog used less space than whiptail. Since then, the difference between the library sizes has grown.

In a test-compile on Debian 7, libslang.so.2.3.2 was 3871404 bytes. Some of that is the symbol table, but the library is still fairly large when considered as an alternative to ncurses:

   text    data     bss     dec     hex filename
1104719  127408 1721400 2953527  2d1137 libslang.so.2.3.2

On the same machine, the ncurses6 library which I use is (with debugging traces compiled in) 521774 bytes — much smaller. Since this a Debian machine, the library size is shown split-up:

   text    data     bss     dec     hex filename
 277170    3568     544  281282   44ac2 libncursesw6.so.6.0
  59536    1516     464   61516    f04c libticw6.so.6.0
 185068   20144    1024  206236   3259c libtinfow6.so.6.0

That amounts to 547002 bytes of text+data. Keep in mind that ncurses does a lot that slang's curses/display functions do not do, including the tracing feature. Debian's current package (for the recently released Debian 9) is smaller:

   text    data     bss     dec     hex filename
 187407    2572     424  190403   2e7c3 libncursesw.so.5.9
  54639    1380     432   56451    dc83 libtic.so.5.9
 150895   14536    2016  167447   28e17 libtinfo.so.5.9

The Debian package is built from the same source, but does not turn on the tracing and debugging features. The ABI difference between ncurses5 and ncurses6 is small enough to ignore here. The total of text+data is 411429 bytes.

If only the curses-related part of the slang library were provided as a separate library, that would be smaller (though less capable) than ncurses. Midnight Commander did that until early 2009.

Considering only the curses-related part of the slang library, starting from the slcurses module and adding its dependencies gives this (a 1290310-byte file, or 292346 bytes of text+data):

   text    data     bss     dec     hex filename
   2286      12      24    2322     912 slcommon.o (ex libslcurses.a)
  10438      28     544   11010    2b02 slcurses.o (ex libslcurses.a)
  26039     348  543420  569807   8b1cf sldisply.o (ex libslcurses.a)
   6788    1452     100    8340    2094 slerr.o (ex libslcurses.a)
    711       4       8     723     2d3 slgetkey.o (ex libslcurses.a)
  65136   34816       0   99952   18670 slischar.o (ex libslcurses.a)
   1648       0      16    1664     680 slkeypad.o (ex libslcurses.a)
   6241       0       8    6249    1869 slmisc.o (ex libslcurses.a)
  18166      16   21296   39478    9a36 slsmg.o (ex libslcurses.a)
   4681       0       0    4681    1249 slutf8.o (ex libslcurses.a)
   2451       4     224    2679     a77 slutty.o (ex libslcurses.a)
  16333   17408       4   33745    83d1 slwcwidth.o (ex libslcurses.a)
   3961     256     107    4324    10e4 slkeymap.o (ex libslcurses.a)
  16530    7832       0   24362    5f2a sllower.o (ex libslcurses.a)
   1661       0     160    1821     71d slsignal.o (ex libslcurses.a)
  10973       0 1130496 1141469  116add slstring.o (ex libslcurses.a)
   7920     100    1024    9044    2354 sltermin.o (ex libslcurses.a)
  19602    7832       0   27434    6b2a slupper.o (ex libslcurses.a)
    673       0       0     673     2a1 slmemcmp.o (ex libslcurses.a)

The bss column counts in the library's runtime size, but does not count in the filesize. It is probably larger than needed e.g., for the display and string modules.

If one discards the curses-related functions and counts just the display, that reduces the size by a third (a 877394-byte file, with 172220 bytes of text+data):

   text    data     bss     dec     hex filename
   2286      12      24    2322     912 slcommon.o (ex libsldisply.a)
  26039     348  543420  569807   8b1cf sldisply.o (ex libsldisply.a)
   6788    1452     100    8340    2094 slerr.o (ex libsldisply.a)
  16530    7832       0   24362    5f2a sllower.o (ex libsldisply.a)
   6241       0       8    6249    1869 slmisc.o (ex libsldisply.a)
   1661       0     160    1821     71d slsignal.o (ex libsldisply.a)
  18166      16   21296   39478    9a36 slsmg.o (ex libsldisply.a)
  10973       0 1130496 1141469  116add slstring.o (ex libsldisply.a)
   7920     100    1024    9044    2354 sltermin.o (ex libsldisply.a)
  19602    7832       0   27434    6b2a slupper.o (ex libsldisply.a)
   4681       0       0    4681    1249 slutf8.o (ex libsldisply.a)
  16333   17408       4   33745    83d1 slwcwidth.o (ex libsldisply.a)

Midnight Commander used to bundle a stripped-down variant of this. Here are sizes from mc-4.6.2 released in September 2007 (with sources from slang 2.0.5):

   text    data     bss     dec     hex filename
   1260      12      24    1296     510 slcommon.o (ex libmcslang.a)
  21029      76   12772   33877    8455 sldisply.o (ex libmcslang.a)
   7070     208      40    7318    1c96 slerr.o (ex libmcslang.a)
    674       4       8     686     2ae slgetkey.o (ex libmcslang.a)
   4746    4168       0    8914    22d2 sllower.o (ex libmcslang.a)
   1455       0       0    1455     5af slmisc.o (ex libmcslang.a)
   1327       0     160    1487     5cf slsignal.o (ex libmcslang.a)
  17496      16   21296   38808    9798 slsmg.o (ex libmcslang.a)
  10449       0   90464  100913   18a31 slstring.o (ex libmcslang.a)
   7355      96    1024    8475    211b sltermin.o (ex libmcslang.a)
   5514    4168       0    9682    25d2 slupper.o (ex libmcslang.a)
   4626       0       0    4626    1212 slutf8.o (ex libmcslang.a)
   2209       4     224    2437     985 slutty.o (ex libmcslang.a)
   7787   17408       0   25195    626b slwcwidth.o (ex libmcslang.a)

Here is a diffstat for comparing the curses-related files for those releases:

_slang.h    | 1406 ----------!!!!!!!!!=======
jdmacros.h  |   53
sl-feat.h   |   74
slang.h     | 2497 ++----------!!!!!!!!!!!!!!!========================
slcommon.c  |  318 ------
slcurses.c  | 1321 --------------------------
slcurses.h  |  380 -------
sldisply.c  | 2951 ++----!!!=================================================
slerr.c     |  766 -----------!!
slgetkey.c  |  320 =====
slinclud.h  |   31
slischar.c  |  161 ---
slkeymap.c  |  702 --------------
slkeypad.c  |  197 ----
sllimits.h  |   96 =
sllower.c   |   17
slmemcmp.c  |   90 -
slmisc.c    |  753 -------------
slsignal.c  |  351 ======
slsmg.c     | 2341 +++----------------!!!!!========================
slstring.c  |  692 --------------
sltermin.c  | 1204 =======================
slupper.c   |   18
slutf8.c    |  841 -----------------
slutty.c    |  618 ===========
slvideo.c   | 2379 !=============================================
slw32tty.c  |  370 !=====
slwcwidth.c |   15
28 files changed, 430 insertions(+), 8193 deletions(-), 2036 modifications(!), 10303 unchanged lines(=)

Demo programs

1996

In 0.99-32, Davis added several demo programs copied from ncurses 1.9.9e (although he did not remove the “color worm” until after 1.2.2, i.e., a couple of years):

Changes since 0.99-31
1. Simple minded curses emulation added.  This should help those migrating
   from curses.  See slang/src/curses/*.c for sample curses demo programs
   that the SLsmg routines can handle.  See also slang/src/slcurses.c for
   some wrapper functions.

Here is a listing from the 0.99-34 tar-ball of the ncurses test-programs which Davis was able to use:

drwxr-xr-x 644/36            0 1996-09-03 01:19 slang/src/curses/
-rw-r--r-- 644/36          427 1996-09-03 01:19 slang/src/curses/README
-rw-r--r-- 644/36          888 1996-09-03 01:19 slang/src/curses/Makefile
-rw-r--r-- 644/36        14116 1996-09-03 01:19 slang/src/curses/battle.c
-rw-r--r-- 644/36         9261 1996-09-03 01:19 slang/src/curses/blue.c
-rw-r--r-- 644/36        28277 1996-09-03 01:19 slang/src/curses/bs.c
-rw-r--r-- 644/36         3129 1996-09-03 01:19 slang/src/curses/firework.c
-rw-r--r-- 644/36         3616 1996-09-03 01:19 slang/src/curses/gdc.c
-rw-r--r-- 644/36         5810 1996-09-03 01:19 slang/src/curses/hanoi.c
-rw-r--r-- 644/36        14181 1996-09-03 01:19 slang/src/curses/knight.c
-rw-r--r-- 644/36         1642 1996-09-03 01:19 slang/src/curses/rain.c
-rw-r--r-- 644/36         3047 1996-09-03 01:19 slang/src/curses/tclock.c
-rw-r--r-- 644/36         3278 1996-09-03 01:19 slang/src/curses/view.c
-rw-r--r-- 644/36         8481 1996-09-03 01:19 slang/src/curses/worm.c

Like many developers, the timestamps reflect the data of release rather than modification. According to the diff-file, he added those files in June 1996 (then again, it may have been earlier for the same reason).

The actual list in ncurses 1.9.9e was longer. Here is a listing from my copy:

drwxr-xr-x tom/users         0 1996-03-29 20:27 test/
-r--r--r-- tom/users      2095 1996-01-20 22:36 test/Makefile.in
-r--r--r-- tom/users       994 1995-11-25 02:11 test/README
-r--r--r-- tom/users      9227 1996-01-18 01:35 test/blue.c
-r--r--r-- tom/users      1816 1995-06-26 23:26 test/bs.6
-r--r--r-- tom/users     28264 1996-01-18 01:35 test/bs.c
-r--r--r-- tom/users      3010 1996-01-18 01:35 test/configure.in
-r--r--r-- tom/users      3129 1995-08-03 01:21 test/firework.c
-r--r--r-- tom/users       482 1995-06-26 23:26 test/gdc.6
-r--r--r-- tom/users      3616 1995-10-02 02:58 test/gdc.c
-r--r--r-- tom/users      5810 1996-03-21 02:53 test/hanoi.c
-r--r--r-- tom/users     14181 1995-09-24 23:05 test/knight.c
-r--r--r-- tom/users      1069 1996-03-12 00:48 test/lrtest.c
-r--r--r-- tom/users     50147 1996-03-12 00:48 test/ncurses.c
-r--r--r-- tom/users      8678 1995-10-02 02:58 test/newdemo.c
-r--r--r-- tom/users      1642 1995-06-26 23:26 test/rain.c
-r--r--r-- tom/users      3047 1996-01-07 22:27 test/tclock.c
-r--r--r-- tom/users     10961 1996-01-18 01:35 test/testcurs.c
-r-xr-xr-x tom/users      2292 1995-10-09 02:00 test/tracemunch
-r--r--r-- tom/users      3278 1995-12-24 01:36 test/view.c
-r--r--r-- tom/users      8481 1996-03-12 00:42 test/worm.c
-r--r--r-- tom/users     30459 1995-06-26 23:26 test/xmas.c

That is, Davis omitted lrtest, ncurses, newdemo, testcurs and xmas. The file battle.c is an earlier version of bs.c, adapted from an earlier release of ncurses (probably 1.9.2d). In short, SLcurses was able to handle ⅔ of the test programs. He probably intended handling newdemo, testcurs and xmas, since the 0.99-38 makefile lists those as targets without providing a rule. Those that did build did not actually work with that version. Testing on a 32-bit Debian6, it seems that window creation was overlooked.

Eventually library fixes were made so that the programs work after a fashion. But while that was going on, ncurses development continued. As I pointed out in 2002 on the UnixOS2 mailing list:

Date:  Wed, 24 Apr 2002 05:06:35 -0400 (EDT)
From:  "Thomas E. Dickey" <dickey at herndon4.his.com>
Subject:  Re:  SLANG sample programs

On Wed, 24 Apr 2002, John Poltorak wrote:

> Has anyone ever managed to compile the sample CURSES programs provided
> with SLANG?

even if they did, there's no guarantee that those "curses" programs would
work with any given curses implementation.  John Davis modified a snapshot
from one of the old ncurses versions to work with slang.  I pointed out to
him a couple of times that this was the case, and that the ncurses test
programs use features which are not in those programs.  (And not all of
them compile on Unix, either).

Of course not all curses implementations work well with those, either.
Last week I was testing with Solaris' implementation of X/Open curses.
It ran the animated ones (e.g., firework) very slowly for some reason.

--
T.E.Dickey <dickey at herndon4.his.com>
http://invisible-island.net
ftp://invisible-island.net

The test-programs in ncurses have been improved in many ways. SLcurses cannot (unless aided) handle any of the current set of test programs:

some changes are minor, e.g., using OK which was omitted from SLcurses, although it defined ERR (hint: all versions of curses define both),
some changes use generally available curses functions which apparently were omitted since the goal of SLcurses was just to compile those demo programs,
some changes were harmless except for SLcurses since SLcurses does less error-checking than most implementations of curses (e.g., allowing an application to move the current cursor position outside a window and dump core), and
some changes were made to the way colors are managed (e.g., using bkgd to avoid tweaks with wattrset).

According to the README file in that directory, Davis made no other changes than to fix compiler warnings. The core dump came from firework, which Davis modified in 0.99-34 to work with SLcurses rather correcting SLcurses. Since ncurses checks for out-of-bounds movement, I had no reason to make that particular change. There were other changes (such as adding calls to napms to worm, as I did later in ncurses, finding that my new machine was too fast).

2017

Testing a library with twenty-year-old versions of test-programs is probably not an effective way to spend time. As of September 2017, ncurses has 72 test-programs, and as noted, color management has changed. Among other things, it would be useful to know how well the slang library copes with current test-programs.

To investigate this, I wrote a script which customizes an environment that helps the configure script see slang as as curses library. There would be no benefit in modifying the configure script to work with slang as it is. The script does this:

creates a temporary directory,
finds where slang is installed,
creates a curses.h which includes slcurses.h, and adds definitions and inline functions to address the deficiencies of SLcurses relative to a curses library,
adds symbolic links to fool the configure script into finding a curses library in that directory,
sets environment variables needed by the configure script, and
runs a subshell (usually ded) from which I can run the configure script, build and test programs.

Using this approach, it is possible to demonstrate a curses-program using (mostly) the slang library. Here is a summary

“working” (no core dumps, etc., although often there are other problems).
non-working (the ncurses examples print a message indicating which part of curses or ncurses is missing).

Program	Working?	Inline...	Problems
background	yes	3	no color, no line-drawing (see screenshots)
blue	yes	0	none
bs	yes	0	none
cardfile		0	requires form and panel libraries
chgat		0	requires chgat function
clip_printw		0	requires vw_printw function
color_set		0	requires color_set function
demo_altkeys		0	uses ncurses extensions
demo_defkey		0	uses ncurses extensions
demo_forms		0	requires form library
demo_keyok		0	uses ncurses extensions
demo_menus		0	requires menu library
demo_new_pair		0	uses ncurses extensions
demo_panels		0	requires panel library
demo_termcap	yes	2	ignores capabilities for meta-mode
demo_terminfo		0	requires terminfo functions
ditto		0	requires delscreen function
dots		0	requires terminfo functions
dots_curses	yes	0	slow, does not catch signals set up before initscr
dots_mvcur		0	requires terminfo functions
dots_termcap	yes	2	does not work with TERM=xterm-256color
echochar	yes	2	very slow, fails to paint whole screen
extended_color		0	uses ncurses extensions
filter		0	requires the filter and wscanw functions
firework	yes	1	none
firstlast	yes	0	required workaround for waddch
foldkeys		0	uses ncurses extensions
form_driver_w		0	requires the formw library
gdc	yes	4	problem with line-drawing
hanoi	yes	1	none
hashtest	yes	0	none
inch_wide		0	requires wide-character curses
inchs	yes	2	see note about wmove
ins_wide		0	requires wide-character curses
insdelln		0	requires winsdelln function
inserts		0	requires winsstr function
key_names		0	requires wide-character curses
keynames	yes	4	none
knight	yes	0	no color
list_keys		0	requires terminfo functions
lrtest	yes	2	addch on box shows only box-lines, also see note
movewindow		0	requires mvderwin, mvwin functions
ncurses	yes	18	no color, omitted 13 of 23 menu items
newdemo	yes	2	none
picsmap	yes	1	no color
railroad	yes	2	assumes screen is 24x80
rain	yes	1	none
redraw	yes	4	terminal mode changes do not work
savescreen		0	requires the screen-dump functions
sp_tinfo		0	requires ncurses extensions
tclock	yes	2	background color does not work
test_add_wchstr		0	requires wide-character curses
test_addchstr	yes	3	newwin gives incorrect window size
test_addstr	yes	1	newwin gives incorrect window size
test_addwstr		0	requires wide-character curses
test_arrays		0	requires terminfo functions
test_get_wstr		0	requires chgat function
test_getstr		0	requires chgat function
test_instr	yes	1	see notes on wmove and waddnstr
test_inwstr		0	requires wide-character curses
test_opaque		0	requires ncurses extensions
test_setupterm		0	requires terminfo functions
test_sgr		0	requires terminfo functions
test_termattrs		0	requires terminfo functions
test_vid_puts		0	requires wide-character terminfo functions
test_vidputs		0	requires terminfo functions
testaddch	yes	1	problems with color (see screenshots)
testcurs	yes	14	omits Pad Test
testscanw	yes	5	SLcurses provides no scanw function
view	yes	6	no color
worm	yes	1	no color
xmas	yes	3	SLcurses provides no overlay function
Total: 72	Working: 33	27	Other modified symbols: 69

Notes:

The Inline... column shows the number of inline-functions used in each program from the header file that allows compiling these programs with SLcurses.

At the time of writing, there are 27 inline functions, as well as 69 simple definitions (not only missing definitions, but including some bug-fixes).

The inline functions include wscanw, which was never implemented in SLcurses (there is an ifdef'd-out implementation which would not have worked), and copywin (which SLcurses defined to a dummy function). The latter is used in xmas to make the tree blink. These chunks in slcurses.h illustrate the problem:

#if 0
/* Why are these functions part of curses??? */
extern int SLcurses_mvwscanw (SLcurses_Window_Type *, unsigned int, unsigned int,
char *, ...);
extern int SLcurses_wscanw (SLcurses_Window_Type *, char *, ...);
extern int SLcurses_scanw (char *, ...);
#define mvwscanw SLcurses_mvwscanw
#define wscanw SLcurses_wscanw
#define scanw SLcurses_scanw
#endif
...
/* Functions that have not been implemented. */
#define copywin(w,v,a,b,c,d,e,f,g) SLcurses_nil()
#define wdeleteln(win) SLcurses_nil()
#define resetty SLcurses_nil
#define savetty SLcurses_nil
#define overlay(u,v) SLcurses_nil()
Without the added definitions, only 51 of the 72 programs successfully compile and link.

From the original set of demo programs, these fail to build if the definitions are omitted:

bs, gdc, tclock, and view.
Several programs set up signal catchers to allow them to print summary information on exit without waiting for a keyboard command to quit. Without that, ncurses would catch quit-signals and cleanup the display.

Other implementations of curses do not catch signals.

While SLcurses catches signals, it differs from ncurses by ignoring existing signal handlers. Also — unlike ncurses — it does not flush its output buffer when catching a signal. If you happen to use curs_set to hide the cursor, you will have to use tput to restore it after quitting an SLcurses program.

One of the bug-fixes replaces SLcurses' curs_set function, which does not flush its output. But working around SLcurses signal handlers would take more work.
Several programs (including those where no problems are noted) have a help-window which will not work with SLcurses, because it uses the pad feature:

chgat, clip_printw, demo_new_pair, edit_field, form_driver_w, inch_wide, inchs, insdelln, movewindow, rain, redraw, savescreen, test_get_wstr and test_getstr.
Unlike SVr4 curses and ncurses, SLcurses will read past the end of a string in its implementation of waddnstr. I noticed this in test_instr, and added a workaround in my script. BSD curses did not have waddnstr. There are other problems in test_instr due to the bugs in SLcurses_waddch.
Several programs run into SLcurses limitations. It has no equivalent for curses' scrollok, apparently assuming that all windows scroll. But it does not succeed in scrolling windows other than the standard screen. Aggravating that, SLcurses_waddch moves the cursor to the top of the window if the row position is past the end of the window.

   if (win->_cury >= win->nrows)
     {
        /* Curses seems to move current position to top of window.  */
        win->_cury = win->_curx = 0;
        return -1;
     }

While the comment is incorrect (curses does not do that), the problem with SLcurses is longstanding, having been introduced in 0.99-35.

It is easy to fall into this trap accidentally because SLcurses' wmove allows a caller to move the cursor outside a window (and returns OK in that case). SVr4 curses and ncurses both check limits, and return ERR when moving the cursor outside a window. 4.3BSD also checked limits (although it did allow the position to be negative).

In inchs, for example, this bug in SLcurses causes the “char:” text near the top of the screen to be overwritten by other text as one moves the cursor around.

The lrtest program draws a box along the screen margins, and moving markers along the box. Due to another bug, the markers on the box-lines do not appear. But due to the bugs in waddch, occasionally the markers wander off the box-lines.

In my script, I added a workaround for wmove which covers most cases aside from mvprintw (done in an internal call of SLcurses). No such workaround is possible for inchs because the bug is within SLcurses_waddch. Its error returns are mostly incorrect; curses applications which rely upon checking the error-return from waddch will not work well with SLcurses.

In some of the original demo programs, color works:

bs, blue, firework, gdc, hanoi, rain, tclock (but see note),

but in others, it does not:

knight, view, worm.

Some of the problems are easy to see in programs designed to test color. Here are comparison screenshots for the background program (ncurses on the left):

ncurses-examples – background example using ncurses

ncurses-examples – background example using slang

and here is a comparison for the testaddch program:

ncurses-examples – testaddch example using ncurses

ncurses-examples – testaddch example using slang

The problems lie in the way color is managed in the slang library.

Color management

SVr4 curses manages color at three levels:

cell (per-character) updates (e.g., passing attributes packed in a chtype),
window attributes (e.g., wattrset) and
the background character (e.g., bkgd).

If an attribute (bold, reverse, underlining, color) is not set in a higher level, but set in the lower level, then the cell is rendered with the combined attributes.

When I first became involved in ncurses development in March 1995, the other developers were unaware of the background character (although the corresponding functions were listed as implemented in the NEWS for ncurses 1.8.6). I changed that:

I had ported add to MS-DOS in 1993.
I used PDCurses to make a white-on-blue color scheme
When I ported that to Linux in 1994, using ncurses, I found that it showed blue only where there was text. Otherwise the screen was black.
Eric Raymond insisted that was the way SVr4 curses worked.
I wrote a demo program which dumped the attributes (and color values) for each cell to prove that SVr4 curses filled the screen with the window attribute.
After some discussion, Raymond gave in, and made changes to the color logic (although in mid-March 1995 he said that his Dell SVr4 did not fill the screen with color).

The relevant NEWS item for this includes these items:

### ncurses 1.8.7 -> 1.8.8
  ...
* The Linux Standard Console terminfo entry is called linux insead of console.
  It also uses the kernel's new method of changing charsets.
  ...
* wbkgd() and wbkgdset() can be used to set a windows background to color.
  wclear()/werase() DO NOT use the current attribute to clear the screen.
  This is the way SVR4 curses works. PDCurses 2.1 is broken in this respect,
  though PDCurses 2.2 has been fixed.
  ...
* many spelling corrections courtesy of Thomas E. Dickey

Over the next few years, ncurses development (including fixes by others) took the vague description of the related background character and (re)implemented that feature, matching Solaris curses. Here is what the relevant slice of code looks like in ncurses 6:

/* Return bit mask for clearing color pair number if given ch has color */
#define COLOR_MASK(ch) (~(attr_t)(((ch) & A_COLOR) ? A_COLOR : 0))

static NCURSES_INLINE NCURSES_CH_T
render_char(WINDOW *win, NCURSES_CH_T ch)
/* compute a rendition of the given char correct for the current context */
{
    attr_t a = WINDOW_ATTRS(win);
    int pair = GetPair(ch);

    if (ISBLANK(ch)
        && AttrOf(ch) == A_NORMAL
        && pair == 0) {
        /* color/pair in attrs has precedence over bkgrnd */
        ch = win->_nc_bkgd;
        SetAttr(ch, a | AttrOf(win->_nc_bkgd));
        if ((pair = GET_WINDOW_PAIR(win)) == 0)
            pair = GetPair(win->_nc_bkgd);
        SetPair(ch, pair);
    } else {
        /* color in attrs has precedence over bkgrnd */
        a |= AttrOf(win->_nc_bkgd) & COLOR_MASK(a);
        /* color in ch has precedence */
        if (pair == 0) {
            if ((pair = GET_WINDOW_PAIR(win)) == 0)
                pair = GetPair(win->_nc_bkgd);
        }
        AddAttr(ch, (a & COLOR_MASK(AttrOf(ch))));
        SetPair(ch, pair);
    }

    TR(TRACE_VIRTPUT,
       ("render_char bkg %s (%d), attrs %s (%d) -> ch %s (%d)",
        _tracech_t2(1, CHREF(win->_nc_bkgd)),
        GetPair(win->_nc_bkgd),
        _traceattr(WINDOW_ATTRS(win)),
        GET_WINDOW_PAIR(win),
        _tracech_t2(3, CHREF(ch)),
        GetPair(ch)));

    return (ch);
}

SLcurses started with a simpler model. Windows were not objects, but just a momentary use of coordinates on the screen. The attrset function set a global value for color which was momentarily overridden by the color passed in per-character calls. Although a window structure was added in 0.99-34:

typedef struct
{
   unsigned int row, col, nrows, ncols;
   unsigned int crow, ccol;
   unsigned short *b;
   unsigned short *buf;
   unsigned short *bufmax;
   unsigned short color;
}
SLcurses_Window_Type;

the library did not clearly distinguish between the window- and character-attributes. The demo programs which work with color set the window attributes. Those that do not set the character attributes.

The initial implementation of SLcurses in 0.99-34 used only window attributes:

int SLcurses_waddch (SLcurses_Window_Type *win, char ch)
{
   unsigned short *b;

   if (win == NULL) return -1;

   b = win->b;
   if (b < win->bufmax)
     {
        *b++ = ch | win->color;
        win->b = b;
        win->ccol++;

        if (win->ccol >= win->ncols)
          {
             win->ccol = 0;
             win->crow++;
          }
     }
   return 0;
}

By 0.99-38 that had been extended to try to take into account character-attributes:

int SLcurses_waddch (SLcurses_Window_Type *win, SLtt_Char_Type attr)
{
   unsigned short *b, ch;
   unsigned short color;

   if (win == NULL) return -1;

   if (win->_cury >= win->nrows)
     {
        /* Curses seems to move current postion to top of window. */
        win->_cury = win->_curx = 0;
        return -1;
     }

   win->modified = 1;

   ch = attr & 0xFF;

   if (attr == ch)
     color = win->color;
   else
     {
        /* hack to pick up the default color for graphics chars */
        if (((attr & A_COLOR) == 0) && ((attr & A_ALTCHARSET) != 0))
          {
             attr |= win->color << 8;
          }
        color = map_attr_to_object (attr);
     }

   if (ch < ' ')
     {
        if (ch == '\n')
          {
             SLcurses_wclrtoeol (win);
             return do_newline (win);
          }

        if (ch == '\r')
          {
             win->_curx = 0;
             return 0;
          }

        if (ch == '\b')
          {
             if (win->_curx > 0)
               win->_curx--;

             return 0;
          }

        /* HACK HACK!!!! */
        if (ch == '\t') ch = ' ';
     }

   if (win->_curx >= win->ncols)
     do_newline (win);

   b = win->lines[win->_cury] + win->_curx;
   *b = ch | (color << 8);
   win->_curx++;

   return 0;
}

and color objects (S-Lang's equivalent to curses color-pairs, with the quirk that video-attributes such as underline were considered a form of color):

static unsigned char Color_Objects[256];

static unsigned int map_attr_to_object (SLtt_Char_Type attr)
{
   unsigned int obj;
   SLtt_Char_Type at;

   obj = (attr >> 8) & 0xFF;

   if (SLtt_Use_Ansi_Colors)
     {
        if (Color_Objects[obj] != 0) return obj;

        at = SLtt_get_color_object (obj & 0xF);

        if (attr & A_BOLD) at |= SLTT_BOLD_MASK;
        if (attr & A_UNDERLINE) at |= SLTT_ULINE_MASK;
        if (attr & A_REVERSE) at |= SLTT_REV_MASK;

        SLtt_set_color_object (obj, at);

        Color_Objects[obj] = 1;
     }
   else obj = obj & 0xF0;

   return obj;

}

but the attempt was unsuccessful, and it has (with few changes after 20 years) several problems:

there is no attempt to combine the window- and character-attributes except when processing graphics characters which have no explicit color set,
originally the WINDOW structure stored attributes and character value in an unsigned short, i.e., 16 bits, which limited the number of color combinations to 256,
color objects, which are entries in a fixed-size table (256 entries, for 16 colors), cannot be modified once created,
the color object values (updated by wattrset, etc.) could represent more combinations since they are stored in the WINDOW structure as a “window color” integer.

As noted, there are a few changes:

Cells in the window are now a structure, to allow for combining characters.
The WINDOW structure stores attributes in the base character for each cell (which being Unicode, uses 24 bits).
At the end of SLcurses_waddch, it calls SLcurses_placechar (introduced in pre2-r2 to support UTF-8), passing the character attribute, which is used only to blank out parts of a multicolumn character. The window color is actually used without modification for building the character-cell in the window.

That explains why some of the demo programs work with color, while others do not: only the window attributes work. As each cell is written, the current window attribute is applied to the cell.

Comparable programs

Davis has rejected the idea of comparing ncurses and slang, e.g., in his comments on comp.lang.c in 1998:

slang vs. ncurses (1998/07/28)

From: davis@space.mit.edu (John E. Davis)
Subject: Re: slang vs. ncurses
Date: 1998/07/28
Message-ID: <slrn6rp9ou.uvt.davis@mygir.davis.net>#1/1
X-Deja-AN: 375611577
References: <6phfg9$119s$1@news.ibm.net.il>
X-Complaints-To: use...@eecs-usenet-02.mit.edu
X-Trace: eecs-usenet-02.mit.edu 901621959 5908 18.75.2.45 (28 Jul 1998 10:32:39 GMT)
Organization: Center for Space Research
User-Agent: slrn/0.9.5.3 (UNIX)
NNTP-Posting-Date: 28 Jul 1998 10:32:39 GMT
Newsgroups: comp.lang.c,comp.lang.c++,comp.unix.programmer

On Mon, 27 Jul 1998 12:02:07 +0300, Ido <i...@lab1.datarecovery.co.il>
wrote:
>I am trying to create a viewer program like mc's.
>I need to display Full 8-bit characters, like it does.
>I tried using ncurses, without success.
>Is there a way to do it with slang (mc uses it instead of ncurses or with
>ncurses)?

   I am not sure what you mean by `full 8-bit characters'.  Are you
trying to draw lines or characters from other non-english languages?
What character set do you want to use?  Iso-Latin-1?  Does you
terminal support characters in the range 128-160?  Or does it
interpret such characters as control sequences?  By default, slang is
8-bit clean and sends characters in the range 160-255 to the terminal
as-is.  This is configurable with the slang variable
SLsmg_Display_Eight_Bit.

>what's the main differance between ncurses & slang?
>what can I do with one, that cannot be done with the other?

   I wrote slang because I did not like the (n)curses programming
interface, and because ncurses under linux was buggy (this is nolonger
true).  All slang's SMG (Screen Management) function and variable
names are prefixed with `SLsmg_', e.g., `SLsmg_refresh', whereas
curses uses a variety of names with no particular naming convention
(e.g., `raw').  SLang's SMG interface is more platform independent and
will also work on MSDOS, OS/2, VMS, Amiga, BEOS, and Unix.  One
advantage of curses is that it has a higher-level interface for panes,
etc...  However, the NEWT package from redhat provides a high-level
windowing library for slang.  In addition, curses is available as a
standard library on most unix systems whereas slang is a separate
library.

>which one is simpler to use?

   They are probably the same.

>which one producess better performance?

   It depends.  For several years, the performace of slang was much
better.  I have heard that the performance of the latest version of
ncurses is just as good as slang.

>btw, sorry about the number of questions in one post. :)

   No problem.  But keep in mind that I am biased since I am the
author of slang.  Of course the same goes for the ncurses maintainer.
You may be better off getting some advice from people who have used
both libraries.  Unfortunately, most people have never heard of slang.

--John

although I pointed out that a comparison was requested:

slang vs. ncurses (1998/8/1):


In comp.unix.programmer John E. Davis <davis@space.mit.edu> wrote:
>    I disagree.  SLang is NOT an implementation of curses, nor is it a
> version of curses.  If you want to make an analogy with computer
> languages, then do not use versions of java for your example.  Instead
> compare perl and fortran, or lisp and C, or C++ and smalltalk.
Back to the original statement.  Someone wanted to know the differences
between ncurses and slang.  One of the chief differences (from the
standpoint of someone who is not already developing an application based on
slang) is that ncurses follows a standard, and slang does not.  Slang
includes a curses emulation, which does not match any other curses
emulation (so that's nonstandard, unless you decide that it's not
supported, in which cases slang is merely incompatible with curses in
general, though most of its screen manipulation functionality overlaps).

-- of course, if slang's curses implementation is not intended for use
   in porting curses applications, this is not a criticism.

> --John

--
Thomas E. Dickey
dickey@clark.net
http://www.clark.net/pub/dickey

Michael Elkins had a different opinion regarding SLcurses, which was reflected in the slang changelog's comment about applications migrating from curses to slang.

A corollary to Davis' comment about his intended use of SLcurses is that he provides no directly comparable slang test-programs in the examples:

keypad is intended to show how slang translates special keys into keycodes.

It appears to need some work, because some keys which it should recognize are not, while others which are different give the same keycode.
pager, according to the README file, should be contrasted with the SLcurses view demo (from 1996). Perhaps.

Actually, this program supports UTF-8 (implemented in ncurses in 2001, implemented in slang in 2008).
smgtest is slang's equivalent to the ncurses test-program.
useropen illustrates how to use readline

Those slang examples total 1525 lines of C, in contrast to 4338 for the curses demos — or 35559 for the complete ncurses-examples.

There are no other test-programs which demonstrate the slang screen-management features:

There are no suitable programs for which one might construct an equivalent test-program using curses to compare the two side-by-side.
Propping up the ncurses-examples with a fixed <slcurses.h> can only go so far without replacing SLcurses entirely.

That being the case, I chose to adapt representative curses programs as needed to provide equivalent functionality using slang:

dots randomly writes colored cells on the screen
view displays a UTF-8 text file, optionally with colors
picsmap displays a picture like “ASCII art”

dots

I began with “dots” thinking that this would show slang at its best, since its focus is on simple low-level operations.

There are several ncurses “dots” programs, using different interfaces: curses, terminfo, terminfo with mvcur and termcap. A straightforward port of the first works easily with slang, whether or not the fixups are applied. However it is much slower than ncurses. Each of the “dots” programs shows a rate when they are interrupted. The rate is expected to differ according to the type of library interface used as well as the number of colors in the terminal description.

For this discussion, I ran each of the tests once for 25 seconds, just to show roughly how the two compared. Here is a table illustrating the results:

Program ncurses slang

Colors 8 16 256 8 16 256

dots 48215 47491 44732 N/A N/A N/A

dots_curses 19558 19457 19937 4951 4570 4870

dots_mvcur 45903 43783 41393 N/A N/A N/A

dots_termcap 49612 49178 45072 55464 broken broken

Program	ncurses	slang
Colors	8	16	256	8	16	256
dots	48215	47491	44732	N/A	N/A	N/A
dots_curses	19558	19457	19937	4951	4570	4870
dots_mvcur	45903	43783	41393	N/A	N/A	N/A
dots_termcap	49612	49178	45072	55464	broken	broken

While dots_curses does run with slang, it does not run properly with 16- or 256-colors because many of the cells are underlined (not part of the program). The colors also appear to be biased, which gave more motivation for developing picsmap (it is easier to see problems with color in a picture).

Here are screenshots for ncurses and slang, using 16 colors:

and using 256 colors:

In the 256-color case, dots_curses is running with the conventional limit of color-pairs (32767) when built with ncurses. The slang library ignores the terminal description in that detail, assuming the square of the number of colors. However (see map_attr_to_object), SLcurses provides for only 256 color combinations in the Color_Objects array.

SLcurses should be able to handle 16 colors (which would make 256 combinations) but appears to have a bug, which accounts for the scattered underlines and the predominance of purple in the screenshot.

The slang library has a non-curses interface which could be used to develop an application, but the programs would be less comparable.

The termcap interface used by dots_termcap also should work equally well for both libraries, and not be constrained by the number of color combinations. But again there is a problem with the slang library. With the 16- and 256-color terminal descriptions, SLtt_tputs prints the message

capability too long

for each attempt to use colors.

Here are screenshots comparing the two libraries with dots_termcap and 256 colors:

The problem with dots_termcap and slang is that SLtt_tgoto computes the length of the capability string, and prints that message if it is longer than 20 bytes.

Here is the setaf string in readable form from xterm-256color:

\E[%?%p1%{8}%<%t3%p1%d%e%p1%{16}%<%t9%p1%{8}%-%d%e38;5;%p1%d%;m

which would be 68 bytes. But the output from tgoto would be

5 bytes if the parameter were in the range 0-7,
6 bytes for color 15, and
12 bytes for color 255

which are all within the intended limit. If the check were removed, slang would work. In a quick test using LD_LIBRARY_PATH, the resulting performance for the three terminal descriptions is roughly the same (although much slower than the system's slang library).

The SLtt_tgoto function appeared in 2.0.0, ifdef'd out. That and related functions were enabled 5 years later in 2.2.3 (December 2010).

One of the reasons I chose dots as an example, was that I supposed slang could be used to compare with a new feature which I was developing. In ncurses 6.1, I developed an extension which uses an integer (typically 32-bits) for color- and pair-values. I used the extension for an existing terminal description and for a new terminal description:

I modified the xterm-256color terminal description to allow ncurses applications to take advantage of the extended range.
I added a new terminal description xterm-direct for direct colors (see the ncurses FAQ).

The xterm-direct description not only uses the extended range for numbers, but also a new extension RGB to simplify initialization (see the user-capabilities manual page).

Because slang reads the (legacy) binary-terminfo files directly (see discussion of Extended Numbers), it will not use the extended information. A slang-specific program should be able to handle rough equivalents of these two terminal descriptions:

for xterm-256color, this is done anyway since slang ignores the capability for the number of color pairs (and for its comparable interfaces the feature does not work properly anyway).
for xterm-direct, slang would rely on having its COLORTERM variable set to “truecolor” or “24bit” (neither value likely to be the name of a terminal description).

For either, those would necessarily be rough equivalents since slang would not be using the ncurses terminal database.

Writing a slang-specific version of dots would be pointless, since no one could tell from looking at the screen whether it used 2⁸, 2¹⁶, or 2²⁴ colors. I considered showing a pair of screenshots to illustrate this, but found that the terminal programs which supposedly support the extended colors were inadequate. While running dots with the xterm-direct terminal description:

gnome-terminal 3.26.1 (2016) used all of the available memory in the machine. I had only allocated a gigabyte (which left some 500 megabytes for the terminal), and it used that up in less than a minute. Revisiting this with two gigabytes, it lasted longer. While dots was running I could see in top that gnome-terminal also was using most of the CPU time.

Normally dots's output rate is limited by the computer's execution speed, along with the relative effort required to update the X display versus formatting escape sequences for moving the cursor and changing colors. For instance, with xterm, the CPU load for the terminal emulator and dots are roughly the same. With both gnome-terminal and konsole, the terminal emulator uses most of the CPU time; with the worst observed performance from gnome-terminal.

Running out of memory when running a simple program like dots shows a serious defect in the terminal emulator, or more likely, the VTE library (since other terminals using this library have the same deficiencies). Debian/testing had the most recent version 0.50 (2017) at the time this section was written.
KDE konsole implemented 256 colors in 2006 (using the xterm control sequences), and introduced a 24-bit direct color feature at the same time (inspired by ITU T.416).

Although its developer had commented as in KDE #107487 on the colon versus semicolon issue (see xterm FAQ), I found that konsole 16.12.0 (2015) does not yet handle colons in the direct-color control sequences (per ECMA-48 and ITU T.416). Because I used the standard syntax in xterm-direct, it cannot be used with konsole.

With a suitably customized konsole-direct terminal description, konsole was able to run dots. It did not use excessive memory, although (like gnome-terminal) it used 70%–80% of the CPU.

Also, both gnome-terminal and konsole showed another performance-related problem. They would frequently “choke” (making no updates to the screen for a second or more — sometimes several seconds), but the reported rate by dots did not reflect this. This is discussed in the xterm FAQ Can I improve the scrolling speed?
I also checked mlterm 3.8.3 (which ran dots very slowly), and stterm 0.6 (2014).

Like konsole, stterm does not support the standard control sequences, but recognizes the original xterm semicolon delimiter rather than colon.

view

My intention with the view program was to see how slang handled UTF-8:

The work done for Lynx was incomplete because it did not align forms as the user would expect.
Tab-expansion would be the first step in making things align on the screen. I had seen problems with the NetBSD curses which did not allow for the actual width of UTF-8 multibyte characters, and wanted to see if slang did this properly.

I started by adapting the view program from ncurses-examples. That has changed a lot since Eric Raymond added it late in 1994:

The initial program read up to 256 lines from a file, and could scroll up and down using cursor keys.
I added a SIGWINCH handler in July 1995, based on Linux and SunOS 4. I used resizeterm in the signal handler, which was known to be “unsafe” but seemed to work well enough. We do not do that anymore.

Based on the reception of my patch, I withdrew that for a while. I created an improved patch in September 1995:
- adding logic to shift the display left/right using cursor-keys (expanding tabs in the file to make this work as expected),
- displaying the screen-size in the status line, and
- showing the line-numbers on the left margin.
I restored my revised patch in September 1996.
A year later, in 1997, I modified view to use KEY_RESIZE. I kept the SIGWINCH handler just for comparison.
In May 2000, I modified view to handle UTF-8. At the time, the wide-character support was experimental.
With help from Sven Verdoolaege in June 2001, ncurses had workable wide-character support. I revised view, using the complex character functions to store and display an array of cchar_t values. I did that to simplify tab-expansion for a file containing UTF-8.

When not configured for wide-characters, view used addchstr with an array of chtype's.
Later in 2001, I
- deprecated the SIGWINCH handler in view,
- modified it to show the name of the last key received,
- added an option to color the screen white-on-blue and
- added a single-step feature along with a clock on the status line.
After that, most of the changes for view, were for maintenance.

A comparable program using slang would have all of the mentioned features, possibly using a similar approach to displaying tabs properly when shifting the text left/right.

I started by trimming down view to build with SLcurses, without wide-character support (calling it view_slcurses). I found that worked easily enough except that color did not work (see the discussion of Color management). I modified that version to work with slang's limitations.

The first version does not support wide-characters, because SLcurses does not have functions for working with wide-characters. Here are screenshots showing what happens with a UTF-8 file (bulgarian-utf8.txt from ncurses-examples):

SLcurses has no documentation other than the source code. I read that, and saw that (as a result of changing data types for version 2) the SLcurses data structures would hold wide-characters. I wrote a second version of view, bundling that with adapted functions from ncurses and was able to display UTF-8. I called that view_slcursesw. Here are two screenshots using that program:

When I revised view in 2001, I did not take into account double-width characters. I modified dialog in 2003 to support UTF-8, and added examples which used double-width characters, but did not further improve view's tab-expansion. Given a mixture of double- and single-width characters, that does not work well. The ncurses library can handle this, but the application has to use the library properly. These two screenshots show the wide-characters test-file shifted left 9 columns in the view program as implemented for ncurses 6.0 (left) and slang (right):

If the text were properly shifted, the “--” characters would line up. They do not, since that version of view assumed all of the characters used a single column. These lines should be different widths because the word “FISH” uses two columns per character in the first line, but the two lines have the same number of characters:

ＦＩＳＨ       -- Cats like ＦＩＳＨ.
fish       -- Cats like fish.

I decided to improve the view program, to use the library for tab-expansion. This does not invalidate the comparison made using view_slcurses and view_slcursesw, since there was no practical way to extend view_slcursesw to handle double-cell characters. The ncurses and slang libraries do their own tab-expansion, and know about double-cell characters. The improved view program does this:

It creates a hidden window, i.e., one that will never be refresh'd.
It writes each line from the input file into the window.
It uses a curses library call to retrieve the content of the line.

The improved view program uses waddch or waddwstr to write the line, and winchnstr or win_wchnstr to read the line depending on whether it is built with the narrow- or wide-character version of ncurses. It has to do some additional work to align the display (see source code if you are curious), since in the latter case view gets an array of complex characters (cchar_t) which can be single- or double-width.

Considering the “additional work” I decided to show how the curses pad feature could be used to simplify things. Here are screenshots with the improved view and the corresponding padview. They are a little different, but there is a reason for that:

If SLcurses had the function for retrieving the content, it might have been interesting to improve the view_slcurses and view_slcursesw programs to use this technique. But it does not have that function. The wrapper which I wrote for testing the ncurses-examples with SLcurses has this, so those test-cases continue to work as well as they can.

Because there was no further improvement possible with SLcurses, and because the results were unsatisfactory, the next step was to construct a slang-specific view program. The pager.c demo in the S-Lang sources was my starting point. The demo does this:

read a file into a linked list
write the data into a “scroller”
respond to cursor-keys and page up/down similarly to view

This additional work was needed:

remove the reverse-video setting for the status line (which slang displayed as cyan-on-black).
eliminate the fixed-size buffers used for reading the file
show line-numbering
show the name of the last key pressed
maintain the status-line clock
optionally color the screen

The scroller is equivalent to a curses pad, with analogous ability for providing a viewpoint for a file which can be moved up/down, left/right. But moving past the demo, there were problems:

The view program shows line-numbers, requiring the scroller viewport to be shifted past the line-numbers.
The scroller uses the tab stops of the screen. This required a workaround: the line-numbers take up a full tab-stop.

The scroller is undocumented. In fact, much of the screen management turns out to be undocumented. Seeing how to make it work properly required reading the library source code, keeping track of the various global- and module-level variables which comprise its state. Any program written that way can break unexpectedly when the developer decides to fix something. Documentation is the developer's contract (or promise) to the user, who knows what the program ought to do.

The difference between the preceding screenshots from view and padview show a “<” in the left side of the window where half of a double-cell character is hinted. The slang library does that — but only at the screen's left margin. With the line-numbers, that (hard-coded) feature of slang is unavailable. I can simulate the feature in ncurses when managing all of the details of the viewport as in the view program, but it would require work to dynamically show the same hint in a pad. Here are screenshots for view_slang, illustrating this difference from slang's viewpoint:

picsmap

The dots program can demonstrate the use of color, but it cannot highlight limitations on the number of colors that ncurses or slang can display. Whether you call it “color pairs” or “color objects” a picture gives a reliable check on the number of combinations that the library can manage, as well as pointing areas for improvement.

A program that generates a set of test-patterns would also achieve this goal, but creating the patterns takes time. Instead, I developed a program which would read certain types of image files and display them:

“.xbm” monochrome files using the X bitmap format
“.xpm” colored (or gray-scale) files using the X pixmap format
“other” files, using ImageMagick convert

The X formats were chosen because they are easily supported in a standalone program, and sample pictures are easy to find. Also, it would be pointless to make the test-program depend solely on a large, complex program. I added a few sample pictures to ncurse-examples for demonstrations. Finding “.xpm” samples with a non-restrictive license was harder; I generated those using gimp.

I called the resulting program picsmap.

There are of course other “ASCII art” viewers. But my goal in writing picsmap was to develop library calls which would extend the range of color pairs and color values. For instance:

Text-applications generally use a fixed (unchanging) palette, i.e., a set of color pairs. Drawing pictures on the screen implies that the palette is not fixed, and also raises the possibility that one may run out of color pairs or want to reuse an existing pair. I added extended functions to simplify management of color pairs beginning in March 2017 (e.g., alloc_pair and find_pair).

The RGB extension solves a different problem (determining the color content for a given color value). Although there are potentially 2⁴⁸ combinations of the 2²⁴ colors in the xterm-direct description, I chose to limit the number of color pairs to 65536 to keep the memory requirements from growing arbitrarily. Still, this limit determines what types of images picsmap and other applications can display.
With 256 colors, a limit of 32768 color pairs is too limited for a program such as dots which can use 65536 combinations.

I addressed that by adding functions which allow larger color-pairs and color-values, beginning in April 2017.

In some cases (e.g., attr_set), there was no need to add a new function because X/Open Curses had set aside a parameter for “future use”
```
The opts argument is reserved for definition in a future version. Currently, the application must
provide a null pointer as opts.
```
Because no other implementation has used the opts parameter, I found it useful (see the manual pages for discussion).
To support those functions, I modified the ncurses library so that the terminal data could hold those values. Some of those values are available using data structures whose binary alignment and size cannot be changed.

Legacy applications such as those which use the data structure implied by <term.h> still use the 16-bit numbers for binary compatibility, but functions such as tigetnum (terminfo) and even tgetnum (termcap) return an integer, so users of the function interface can use the larger values.

I addressed the header-file issues by making TERMINAL (the container for those capabilities) an opaque object, and adding a corresponding set of capabilities inside that object which use the larger values.
At the same time, I made changes to tic and infocmp to handle these larger numbers.

Because the work would be incomplete without documenting and publishing a comparison (such as this), I maintained the final set of changes in a branch. There were no other developers involved in any of this work.
I also modified or wrote new terminal descriptions such as xterm-256color (increasing the number of pairs), and xterm-direct (direct-colors using 24-bits).

While making these changes, I kept in mind compatibility:
- Older versions of ncurses tic will limit the numbers to the maximum value for signed 16-bit integers (32767).
- When configured for wide-characters, ncurses 6.1 tic uses a new binary format for terminal descriptions containing a number larger than 32767. When configured for narrow characters, the ncurses 6.1 library can read this format, but limits numbers to 32767.
- The new/modified terminal descriptions use the extended-number feature.
- Other implementations of curses (Solaris, NetBSD) do not handle these extended descriptions in the same way as ncurses. Both set the out-of-range numbers to zero. However, neither Solaris SVr4 curses or NetBSD 7 curses supports 256 (or more) colors: Solaris curses lacks the bits, while NetBSD curses malfunctions, including core-dumps, when using xterm-88color or xterm-256color.

For development and routine testing, I collected a variety of images (mostly JPEGs, but also GIFs, PNGs, and of course X bitmap and pixmap files). About a quarter of the files are grayscale (no more than 256 shades of gray).

I wrote a script which runs picsmap first using TERM=xterm-256color and then using TERM=xterm-direct, over the test suite. The script supports options (using xterm's control sequences) which resize the terminal to a fairly large screen, and use xterm's unreadable (2x1 pixels) font, giving a passable result. I chose to use a fixed window size (128x256) for most tests because it is close to the maximum useful character dimensions in the smallest readable font on my monitor (116x254). Doing that also keeps the color palette from growing too large to render the scaled images.

For demonstration purposes, of course, it is not practical to present the entire test suite. But a few examples which demonstrate differences between ncurses and slang, or show the limitations of picsmap itself are suitable.

Here are screenshots for the two terminal settings using a diagram from the Color Dialog Box documentation:

The visual effect using xterm-direct is an illusion, because your eyes cannot distinguish the pixels in the illustration. Switching to a larger font makes it possible, as shown here. If you click on the image, you should be able to see the individual cells which correspond to the pixels in the preceding figure:

Only a small part of picsmap (100 lines of 1700) actually uses curses. The rest reads image files, sets up data structures, etc. After making it work properly with ncurses, I modified it to allow building it with a display-driver, e.g., one written for slang.

Developing the display driver required reading the library source-code because S-Lang's documentation does not cover any of the necessary features. I made two versions of the display driver with slang:

S-Lang has two ways to set the colors in a color object. First, I tried SLtt_set_color_fgbg, which (like init_pair) accepts numbers for the foreground and background colors.

Because the slang library cannot read the terminal description, it falls back to hard-coded behavior, assuming the 8 ANSI colors. There is a variable SLtt_Use_Ansi_Colors which a caller can use to override the number of colors. However, it is applied too late to be useful in this scenario. Also, there is no way (without modifying the library) to change the escape sequences which it uses for color. Those are hard-coded, private variables.

I modified the library:

--- sldisply.c.orig     2016-11-24 21:32:42.000000000 -0500
+++ sldisply.c  2017-11-17 20:29:49.000000000 -0500
@@ -2472,12 +2472,38 @@

    if (ct != NULL)
      {
-       /* Allow easy mechanism to override inadequate termcap/terminfo files. */
-       SLtt_Use_Ansi_Colors = 1;
+        /* yet another hard-coded workaround... */
#if SLTT_HAS_TRUECOLOR_SUPPORT
-       if ((0 == strcmp(ct, "truecolor")) || (0 == strcmp(ct, "24bit")))
-         Has_True_Color = 1;
+        if ((0 == strcmp(ct, "truecolor")) || (0 == strcmp(ct, "24bit")))
+          {
+            Has_True_Color = 1;
+           /*
+            * This should also check if the "Terminfo" variable is null,
+            * but that is declared further in the file.
+            */
+            if (0 == SLtt_Use_Ansi_Colors)
+              {
+                /*
+                 * These strings are from the "xterm-direct" entry written
+                 * for ncurses 6.1 and xterm patch #331:
+                 */
+                Color_Fg_Str = "\033[%?%p1%{8}%<%t3%p1%d"
+                               "%e38:2:%p1%{65536}%/%d"
+                                     ":%p1%{256}%/%{255}%&%d"
+                                     ":%p1%{255}%&%d%;m";
+                Color_Bg_Str = "\033[%?%p1%{8}%<%t4%p1%d"
+                               "%e48:2:%p1%{65536}%/%d"
+                                     ":%p1%{256}%/%{255}%&%d"
+                                     ":%p1%{255}%&%d%;m";
+                /*
+                 * ...and this is the maximum value for the colors capability
+                 * from that terminal description -TD
+                 */
+                Max_Terminfo_Colors = 0x1000000;
+              }
+          }
#endif
+       SLtt_Use_Ansi_Colors = 1;
      }

    if (SLtt_Use_Ansi_Colors)

By doing that, and setting the COLORTERM variable to “24bit” and setting COLORTERM_BCE, as well as setting the number of colors based on TERM, I obtained an S-Lang port of picsmap.

The author's misuse of the term "truecolor" is another issue with the library mentioned in the ncurses FAQ Why only 16 (or 256) colors? and my quotes from S-Lang's source code do not make its author(s) more truthful. For discussion purposes, “24bit” is descriptive, but “direct-color” is preferred.
Modifying the library to add more hard-coded behavior is unsatisfying. S-Lang's other method for setting the colors in a color object requires the caller to pass the information as strings. I had preferred the other call, because the string interface is less efficient: the caller must format a string with the color information, and slang has to decode it.

To set a color by name (i.e., string with the slang library, use SLsmg_set_color. By reading the source code, you may discover that if the string begins with #, slang decides that the string is an RGB value in hexadecimal form, for 24-bits, and calls this function to decode the string into a color number:

static int parse_true_color (const char *color, SLtt_Char_Type *c)
{
   SLtt_Char_Type rgb;
   unsigned int i;
   int h[6];

   i = 0;
   while (i < 6)
     {
        if (-1 == (h[i] = parse_hex_digit (color[i])))
          return -1;
        i++;
     }
   if (color[i] != 0)
     return -1;

   if (i == 3)                         /* RRGGBB */
     rgb = (h[0] << 20) | (h[0] << 16) | (h[1] << 12) | (h[1] << 8) | (h[2] << 4) | h[2];
   else if ((i == 6) && (color[i] == 0))
     rgb = (h[0] << 20) | (h[1] << 16) | (h[2] << 12) | (h[3] << 8) | (h[4] << 4) | h[5];
   else
     return -1;

   *c = rgb | TRUE_COLOR_BIT;
   return 0;
}
#endif

and later, it converts a color number into three components (red, green and blue) and prints the result using sprintf and tt_write:

#if SLTT_HAS_TRUECOLOR_SUPPORT
# if defined(__GNUC__)
#  pragma GCC diagnostic ignored "-Wformat-nonliteral"
# endif
static void write_truecolor (const char *fmt, SLtt_Char_Type c)
{
   char tmpbuf[32];
   int r, g, b;

   r = (int)((c>>16) & 0xFF);
   g = (int)((c>>8) & 0xFF);
   b = (int)(c & 0xFF);

   sprintf (tmpbuf, fmt, r, g, b);
   tt_write_string (tmpbuf);
}
# if defined(__GNUC__)
#  pragma GCC diagnostic warning "-Wformat-nonliteral"
# endif
#endif                                 /* SLTT_HAS_TRUECOLOR_SUPPORT */

Seeing that raises the question: since slang provides a tparm clone called “tt_format” which could have been used to replace the entire “write_truecolor” function, what particular advantage is gained by this hard-coded approach?

For instance, a more general approach might use this additional change:

--- sldisply.c.orig     2016-11-24 21:32:42.000000000 -0500
+++ sldisply.c  2017-11-23 14:54:48.759388803 -0500
@@ -232,10 +232,6 @@

static SLCONST char *Color_Fg_Str = "\033[3%dm";
static SLCONST char *Color_Bg_Str = "\033[4%dm";
-#if SLTT_HAS_TRUECOLOR_SUPPORT
-static SLCONST char *Color_RGB_Fg_Str = "\033[38;2;%d;%d;%dm";
-static SLCONST char *Color_RGB_Bg_Str = "\033[48;2;%d;%d;%dm";
-#endif
static SLCONST char *Default_Color_Fg_Str = "\033[39m";
static SLCONST char *Default_Color_Bg_Str = "\033[49m";

@@ -1614,27 +1610,6 @@
    last_i = i;
}

-#if SLTT_HAS_TRUECOLOR_SUPPORT
-# if defined(__GNUC__)
-#  pragma GCC diagnostic ignored "-Wformat-nonliteral"
-# endif
-static void write_truecolor (const char *fmt, SLtt_Char_Type c)
-{
-   char tmpbuf[32];
-   int r, g, b;
-
-   r = (int)((c>>16) & 0xFF);
-   g = (int)((c>>8) & 0xFF);
-   b = (int)(c & 0xFF);
-
-   sprintf (tmpbuf, fmt, r, g, b);
-   tt_write_string (tmpbuf);
-}
-# if defined(__GNUC__)
-#  pragma GCC diagnostic warning "-Wformat-nonliteral"
-# endif
-#endif                                /* SLTT_HAS_TRUECOLOR_SUPPORT */
-
static void write_attributes (SLtt_Char_Type fgbg)
{
    int unknown_attributes;
@@ -1688,8 +1663,8 @@
             if (fg0 == SLSMG_COLOR_DEFAULT)
               tt_write_string (Default_Color_Fg_Str);
#if SLTT_HAS_TRUECOLOR_SUPPORT
-            else if (IS_TRUE_COLOR(fg0))
-              write_truecolor (Color_RGB_Fg_Str, fg0);
+            else if (IS_TRUE_COLOR(fg0) && Max_Terminfo_Colors >= 0x1000000)
+              tt_printf (Color_Fg_Str, fg0 & 0xffffff, 0);
#endif
             else
               tt_printf (Color_Fg_Str, COLOR_ARG(fg0, Is_Fg_BGR), 0);
@@ -1701,8 +1676,8 @@
             if (bg0 == SLSMG_COLOR_DEFAULT)
               tt_write_string (Default_Color_Bg_Str);
#if SLTT_HAS_TRUECOLOR_SUPPORT
-            else if (IS_TRUE_COLOR(bg0))
-              write_truecolor (Color_RGB_Bg_Str, bg0);
+            else if (IS_TRUE_COLOR(bg0) && Max_Terminfo_Colors >= 0x1000000)
+              tt_printf (Color_Bg_Str, bg0 & 0xffffff, 0);
#endif
             else
               tt_printf (Color_Bg_Str, COLOR_ARG(bg0, Is_Bg_BGR), 0);

replacing 28 lines of source with 4.

When testing picsmap, I found that this reduced the number of characters sent to the terminal by about a quarter of one percent. The variation in time (elapsed, user or system) from one test run to the next is far larger than that. Someone might be interested in benchmarking slang to prove whether sprintf is faster than tt_printf, but I did not find the hard-coded approach to be an improvement.

I was reminded of slang's string method because picsmap represents the default color as -1. But the slang library does not use that value for “default” it uses a hidden value that can be used only via the interface that specifies colors by name (strings).

Passing a -1 via SLtt_set_color_fgbg caused the library to show pure white, because the library checks one bit to decide if it is “truecolor”, and given that, all of the bits for red/green/blue are also set. In other words, the slang library is maintaining two types of colors: “ANSI” (indexed color) and RGB (direct color).

While developing ncurses, I have not found it useful to make that distinction since slang relies upon the COLORTERM environment variable to make the scheme work (and Davis' repeated assertions that the purpose of that variable is solely to work around defective terminal descriptions make other interpretations a moot point). Instead, in ncurses I chose to provide a new extended capability RGB for terminal descriptions which tells the library if the terminal supports that type of encoding, and how the bits are allocated. The 24-bit/3-byte arrangement is a useful special case, but not the only way a terminal might support the feature.

As a tradeoff, rather than modify ncurses to support two (or more) types of color or hard-code behavior to support (as implied by the emacs patch mentioned in the FAQ) “setf24” or “setb24” , I chose to extend the range of color values. However that introduces a problem with applications that use indexed colors. To accommodate those I made the xterm-direct terminal description a hybrid of indexed- and direct-colors:

for values 0 through 7, it uses the “ANSI” control sequences, while
for other values, it uses the 3-byte direct-color sequence introduced by Konsole.
the number of colors is 2²⁴ while the number of color pairs is 2¹⁶

That is, black is the same in either encoding, but values 1-7 (which represent an insignificant part of the 24-bit range) provide the conventional ANSI colors. Little would be gained by doing the same thing for xterm's 256-color encoding (applications which rely upon specific color values in its “color cube” are outside the scope of ncurses); I chose to not provide an example in this discussion.

Nothing in the foregoing precludes the possibility of extending ncurses's API to support two (or more) color models. However (as of November 2017), no one has presented a valid scenario where that is necessary.

After I made those fixes to slang, the two display drivers using slang were able to display most of the images in my test suite.

Here is a pair of images illustrating the improvement made for the first program, picsmap_slang, using the xterm-direct terminal description:

Without the fixes made for its built-in hard-coded defaults, slang will handle monochrome images, and a few color images (no grayscale). For instance, it handles these cases:

Even with those fixes, the slang-based versions of picsmap will not successfully display some images because the size of its color objects table is smaller than the limit which I chose for ncurses 6.1. Here is a pair of screenshots (ncurses on the left as usual) illustrating what happens:

One might not suspect it from viewing the (good) image, but it uses many colors. The log from picsmap says this

opened /usr/share/X11/rgb.txt
opened whirlpoo.gif
...opening pipe to identify "whirlpoo.gif" 2>/dev/null
...opening pipe to convert -resize 840x900\! -thumbnail 256x "whirlpoo.gif" -define txt:compliance=SVG txt:- 2>/dev/null
...using 57027 colors
Number of colors versus number of cells
   50013 colors (87.7%) in 63130 cells (90%)
   56326 colors (98.8%) in 69443 cells (99%)
   56957 colors (99.9%) in 70074 cells (99.9%)
   57020 colors (100.0%) in 70137 cells (99.99%)
   57027 colors (100.0%) in 70144 cells (100.000%)
Allocations:
  777748 file
   12096 name
       0 list
       0 data
      96 head
  458296 pair
  561152 cell
  917504 window

The original GIF file is 840x900, and the screenshot only 262x284 pixels (74408 cells). But ImageMagick converted the image, assigning a different colors for 76% of the cells. That adds up.

If I had created a larger image, the converter would have generated even more colors because it interpolates the values between the known data. Given enough room, the number of colors stretches, limited only by the 24-bit representation.

Converting images is inefficient, but storing the converted images would be costly as well. Here are some numbers from a test-run:

My test-suite with 78 files, used 14651903 bytes of storage.
While running the test suite, picsmap sent 93000327 characters to the screen (6.235 times the size of the input files).

The test script uses a 128x256 pixel window. Bigger windows would use more characters (and more colors). A full-screen window would use 20 times as many characters (and colors).
To avoid creating temporary files, picsmap reads the the output from convert via a pipe. If it stored the data as intermediate files, it would use 121462299 bytes (8.28 times the size of the original files).

Generating those intermediate files took this much time:

66.01user 6.07system 0:53.52elapsed 134%CPU (0avgtext+0avgdata 83388maxresident)k
0inputs+237248outputs (0major+817587minor)pagefaults 0swaps

Running the actual test-suite (which includes the image conversion) took this much time:

163.50user 6.02system 2:20.18elapsed 120%CPU (0avgtext+0avgdata 83472maxresident)k
0inputs+248outputs (0major+2018535minor)pagefaults 0swaps

While running the test suite, picsmap used more than 250Mb of memory (not all at the same time), according to valgrind:

==10311== HEAP SUMMARY:
==10311==     in use at exit: 3,474,401 bytes in 417 blocks
==10311==   total heap usage: 920,588 allocs, 920,171 frees, 267,293,613 bytes allocated

and (the second pass using TERM=xterm-direct):

==11024== HEAP SUMMARY:
==11024==     in use at exit: 3,467,227 bytes in 416 blocks
==11024==   total heap usage: 920,582 allocs, 920,166 frees, 267,220,932 bytes allocated

The test showed me that there was little difference in ncurses memory usage for the two terminal descriptions.

I made all of the screenshots using xterm (patch #131), because it has low overhead. If I had used gnome-terminal, I would have needed a bigger machine. Having seen its poor performance with dots, I took the time to check it with the picsmap test suite:

The machine (with 2Gb of memory) ran out of memory before completing the second pass of the the test-suite, gnome-terminal having consumed 1.3Gb of memory and 800Mb of swap.
In contrast, xterm's memory increased by about 100Kb (a few orders of magnitude smaller).

Color gimmicks

The dots and picsmap programs show what the current ncurses and slang libraries can do with interesting uses of many colors.

Those are libraries. There are several hard-coded scripts (and a few programs) using many colors to display images on a terminal, but those are not topical here because their design constraints are radically different. Both ncurses and slang hold the complete image data in memory before sending it to the terminal. Scripts generally do not do that.

Seeing the poor performance of gnome-terminal caused me to wonder what possible use is being made with slang with COLORTERM=24bit. It could not be either animation or pseudographics. The answer lies in Midnight Commander.

Rather than continue copying the look & feel of Norton Commander, the developers of Midnight Commander several years ago varied the look slightly by allowing users to change the white-on-blue to different shades of color, imitating vim's color schemes (see documentation).

In short, it is a static, very limited use of colors in comparison with either dots or picsmap:

The program loads a color palette (or in their jargon, a “skin”) once, at startup.
The sample “skins” which come with the program use a palette of about 150 colors.

Just as a demo, here is a change which could be made to Midnight Commander using ncurses' extended color pairs:

# patch by Thomas E. Dickey <dickey@invisible-island.net>
# created  Wed Dec  6 09:26:36 UTC 2017
# ------------------------------------------------------------------------------
# color-ncurses.c |   25 +++++++++++++++++++------
# tty-ncurses.h   |    9 +++++++++
# 2 files changed, 28 insertions(+), 6 deletions(-)
# ------------------------------------------------------------------------------
Index: lib/tty/color-ncurses.c
--- old/lib/tty/color-ncurses.c     2017-03-04 17:51:38.000000000 +0000
+++ new/lib/tty/color-ncurses.c     2017-12-06 09:17:25.000000000 +0000
@@ -104,9 +104,9 @@
                                 int fg1, int bg1, int fg2, int bg2, int attr)
{
     if (has_colors () && !mc_tty_color_disable)
-        init_pair (mc_color_pair->pair_index, fg1, bg1);
+        InitPair (mc_color_pair->pair_index, fg1, bg1);
     else
-        init_pair (mc_color_pair->pair_index, fg2, bg2);
+        InitPair (mc_color_pair->pair_index, fg2, bg2);
     mc_tty_color_save_attr (mc_color_pair->pair_index, attr);
}

@@ -196,7 +196,7 @@
             }
         }

-        init_pair (mc_color_pair->pair_index, ifg, ibg);
+        InitPair (mc_color_pair->pair_index, ifg, ibg);
         mc_tty_color_save_attr (mc_color_pair->pair_index, attr);
     }
}
@@ -206,7 +206,11 @@
void
tty_setcolor (int color)
{
-    attrset (COLOR_PAIR (color) | color_get_attr (color));
+    void *opts = NULL;
+#if NCURSES_EXT_NUMBERS
+    opts = &color;
+#endif
+    attr_set (color_get_attr (color), color, opts);
}

/* --------------------------------------------------------------------------------------------- */
@@ -238,9 +242,18 @@
gboolean
tty_use_truecolors (GError ** error)
{
-    /* Not yet supported in ncurses */
-    g_set_error (error, MC_ERROR, -1, _("True color not supported with ncurses."));
+#if NCURSES_EXT_NUMBERS
+    /* mc assumes that xterm-256color can mean "true color" */
+    if (COLORS > 256 || tty_use_256colors ())
+    {
+       return TRUE;
+    }
+    g_set_error (error, MC_ERROR, -1,
+                 _("This program expects at least 256 colors."));
+    return FALSE;
+#else
     return FALSE;
+#endif
}

/* --------------------------------------------------------------------------------------------- */
Index: lib/tty/tty-ncurses.h
--- old/lib/tty/tty-ncurses.h       2017-03-04 17:51:38.000000000 +0000
+++ new/lib/tty/tty-ncurses.h       2017-11-29 10:23:25.000000000 +0000
@@ -25,6 +25,15 @@
#define NCURSES_CONST const
#endif

+#if defined(NCURSES_VERSION) && (NCURSES_VERSION_PATCH > 20170401)
+#define NCURSES_EXT_NUMBERS 1
+#define ValidColor(c)  (((c) < 0) ? -1 : ((c)%COLORS))
+#define InitPair(n,f,b) init_extended_pair(n,ValidColor(f),ValidColor(b))
+#else
+#define NCURSES_EXT_NUMBERS 0
+#define InitPair(n,f,b) init_pair(n,f,b)
+#endif
+
/*** typedefs(not structures) and defined constants **********************************************/

/*** enums ***************************************************************************************/

The configure script also needs work (since it makes too many assumptions about the names of the ncurses header files). I worked around that using a script to modify the generated config.status file.

In the patch:

I redefined init_pair to init_extended_pair.
Then I then noticed that mc passes out-of-range values (an extra digit makes it way into the foreground and background values). The slang library ignores that; ncurses checks for errors.
I worked around the problem in mc by the ValidColor macro, limiting the values to either -1 (default color) or 24-bits (RGB values).

The corresponding code for slang relies upon the COLORTERM environment variable, and assumes that the terminal description identified by TERM is “xterm-256color” if COLORTERM is used.

Here are a few screenshots to illustrate (using xterm and ncurses), with 256 colors on the left, and direct colors on the right:

I was surprised by the difference, and on investigation found that it is a deficiency in Midnight Commander. Unlike xterm and picsmap, it has no provision for showing a 24-bit color scheme on a 256-color display (or a 256-color scheme on a 24-bit color display). The xterm 256-color palette is very stable, copied by developers of other terminal emulators. Given the tendency of Midnight Commander developers to hard-code features, one might expect to see the palette as a table in Midnight Commander's source code. No such table exists. Nor is there (as in picsmap) a provision for reading a color palette from a file. All of this is old technology, well understood (and used) by experienced developers.

Most of Midnight Commander's color schemes (called “skins” for some reason) use 256-colors, and refer to colors by number (the index). The color parameter passed to the ncurses or slang interfaces is an indexed color (palette) value for 256-color skins, and a direct color (RGB) value for 24-bit colors. As a result, the screenshot on the left uses the low-order eight bits of a 24-bit RGB value.

Comparing that particular color scheme (seasons-summer16M.ini) with xterm's indexed- (approximation) and direct-colors, would give too little difference for most readers. The best contrast for discussion purposes is seasons-winter16M.ini, shown here with xterm using indexed colors on the left, and direct colors on the right:

In both screenshots, TERM is xterm-direct, to work within the limitations of Midnight Commander. Because the colors are distinct, xterm's approximation mapping 24-bit colors into its 8-bit color palette usually is as readable as the original scheme.

Getting useful screenshots using slang is complicated by the fact that in my development environment I have more than one terminal database: the current one from routine builds of ncurses (in /usr/local/ncurses/share/terminfo) and the prepackaged one.

Setting the TERMINFO variable to point to the current database leaves slang finding that the first entry for xterm-256color is “defective” and falling back to the prepackaged entry.
On the other hand, if I ask for a new TERM such as “xterm-direct” then slang cannot read that, falling back to ANSI colors.

Also, of course, because Midnight Commander relies upon having COLORTERM set to use more than 256 colors, it can only see the number of colors (8) used by slang in this mode.

Here are screenshots to show the behavior in the two modes:

Influences

The ncurses and slang libraries have had some influence on each other. From my point of view:

slang uses ncurses's terminal database,
although Davis has hard-coded the most commonly used features, while
occasionally I have added features to ncurses to deal with a nuisance.

For the record, ncurses and slang, even with Davis' frequent resort to hard-coding, are more closely related than some purely hard-coded applications that you may encounter. As an example of that mindset, bear in mind Davis' response to Bill Spitzak on comp.os.linux.development.apps in the thread DOS-functions to Linux... in 1998:

From: davis@space.mit.edu (John E. Davis)
Subject: Re: DOS-functions to Linux...
Date: 1998/05/22
Message-ID: <slrn6m8fju.6m0.davis@mygir.davis.net>#1/1
X-Deja-AN: 355510504
References: <3556FDBA.14F76BD4@fiol.brock.dk> <65a_9805121308@antares.antar.com>  <3559A45C.5D211C4F@cinenet.net>
Organization: Center for Space Research
Newsgroups: comp.os.linux.development.apps

On Wed, 13 May 1998 06:47:08 -0700, Bill Spitzak <spitzak@cinenet.net>
wrote:
>Modern terminals and emulators all use the same escape sequences, so you
>don't have to use those libraries.

One of the reasons that I do not recommend this is that it will not
provide you with the optimization that slang or curses provides.

Color model

As I related in the section on Screen Management, I started changes in ncurses to make its color-handling (more) consistent with PDCurses. I had ported my adding machine to use PDCurses in fall 1993, and about a year later, ported it to Linux (Slackware). Through some experimentation, I surmised that the problem with ncurses was that it was not using the bce feature to fill the screen with color. That is, ncurses should use the feature if it existed, but even without that, it should fill the screen with the color I specified. The fine distinction between window and background attributes had not come up yet.

Choose sides

Davis became interested in background color around the same time. From his comments and reviewing the changes made to slang, it seems that the Linux console was his first encounter with a terminal that could do color. (The combined DOS and OS/2 video driver is not the same, really, although it gives some clue to Davis' expectations).

Google groups has part of a thread discussing this in August 1994 (apparently all of Davis' postings were deleted, leaving only the replies):

color ansi terminals??

At that point, Davis already had an opinion not about how a library should accommodate different terminals, but what terminals should be used:

In article <1994Aug15....@pacific.mps.ohio-state.edu>, da...@pacific.mps.ohio-state.edu writes:
|> In article <jmg.77...@dxcoms.cern.ch>, j...@dxcoms.cern.ch (J.M. Gerard)
|> writes:
|>  : I haven't tried either of these, but I have tried DECterm, NCDterm,
|>  : colterm, mterm, QVT on a PC and so on. The number of variations is
|>  : astounding. In the end I gave up touching the background colour: just
|>  : told the user to select what (s)he wants!
|>
|> For my JED editor, I warn the user that if using MS-Kermit to set the
|> default background color of the terminal emulator to the default background
|> desired for the editor. This restriction is not necessary when using the
|> Linux console which is why I think that its handling of the situation is
|> better and should be adopted as a model for others.

and

In article <1994Aug19.214529.14786@pacific.mps.ohio-state.edu>,
 davis@pacific.mps.ohio-state.edu writes:
 |>
 |> In article <1994Aug18.174019.868@hpcvusn.cv.hp.com>, daves@hpcvusj.cv.hp.com
 |> (Dave Serisky) writes:
 |>  : Curses provides support for this difference.  According to the terminfo
 |>  : man page, "Some terminals (for example, most color terminal emulators
 |>  : for PCs) erase areas of the screen with current background color.  In
 |>  : such cases, bce (background color erase) should be defined."  I did look
 |>  : through the terminfo database of a unix vendor that supports color
 |>  : curses and found that by about 10:1, the majority of the terminfo
 |>  : entries did not specify bce.  That is to say that the majority of color
 |>  : terminals do not erase in the current background color.
 |>
 |> This is very unfortunate.  Suppose my display is 24x80. I would have to
 |> write 1920 spaces just to erase the screen in a certain color!

On occasion, I tried making a similar point to Davis. Here is one of his responses:

From davis@space.mit.edu Thu Sep  5 10:59 EDT 1996
Received: from space.mit.edu (SPACE.MIT.EDU [18.75.0.10]) by mail.Clark.Net (8.7.3/8.6.5) with SMTP id KAA15093 for <dickey@clark.net>; Thu, 5 Sep 1996 10:59:10 -0400 (EDT)
Received: from aluche.mit.edu by space.mit.edu AA21391; Thu, 5 Sep 96 10:59:08 EDT
Received: (from davis@localhost) by aluche.mit.edu (8.7/8.7) id KAA16362; Thu, 5 Sep 1996 10:59:08 -0400
From: "John E. Davis" <davis@space.mit.edu>
Date: Thu, 5 Sep 1996 10:59:08 -0400
Message-Id: <199609051459.KAA16362@aluche.mit.edu>
To: dickey@clark.net
Posted-To: comp.os.linux.misc
Subject: Re: why colors or different in xterm than in console ?
References: <3224F9F0.7CE95032@public.uni-hamburg.de> <322D5CA2.5116FA4A@donet.com> <slrn52rito.fc4.davis@aluche.mit.edu> <50m8bl$h26@clarknet.clark.net>
Organization: Center for Space Research
Reply-To: davis@space.mit.edu
Content-Type: text
Content-Length: 1268
Status: RO

[I have also posted the following text to Usenet.]

On 5 Sep 1996 10:00:21 GMT, T.E.Dickey <dickey@clark.net>
wrote:
 : by way of explanation: color_xterm (and ansi_xterm) don't support
 : "background color erase" (which is an attribute that's testable by conforming
 : termcap and termino applications ;-).  bce (for short) denotes whether
 : the terminal retains background color during clear screen/line operations.

This type of color implementation is not only inefficient, it also
looks ugly during screen updating.  Consider an application that tries
to keep a blue background of one section of the display whose default
background color is red.  If for some reason it erases to the end of
the line, the line will appear to be red and the application will then
have to set the color to blue and output enough spaces to change the
background back to blue.  As a result, the user will see a momentary
flash of red.

Unfortunately it is a sad fact that most color terminals and emulators
use this implementation.  Fortunately the Linux console does not (was
that by accident or design?).
--
John E. Davis                   Center for Space Research/AXAF Science Center
617-258-8119                    MIT 37-662c, Cambridge, MA 02139
http://space.mit.edu/~davis

Davis continued to promote the “Linux” color model as the only acceptable behavior for slang, as shown in a thread on comp.terminals in May 1999:

Newsgroups: comp.os.linux.misc,comp.terminals
Organization: Columbia University
Message-ID: <7gpgn4$s1h$1@newsmaster.cc.columbia.edu>
References: <slrn7idjui.vi4.timsuth@fragolian.zone>
<IX3W2.230$qh4.14590@iad-read.news.verio.net>
<slrn7iioqc.kf.timsuth@fragolian.zone> <slrn7irqpk.3ac.davis@mygir.davis.net>
Date: 5 May 1999 13:22:44 GMT
From: Jeffrey Altman <jaltman@watsun.cc.columbia.edu>
Subject: Re: 'screen' and dselect/lynx/mutt/slrn (terminfo?)
...
In article <slrn7irqpk.3ac.davis@mygir.davis.net>,
John E. Davis <davis@space.mit.edu> wrote:
: On 30 Apr 1999 08:20:04 GMT, Tim Sutherland <timsuth@digicron.moof>
: wrote:
: >So the "bce" problem is a slang problem then?
:
: Slang does not support terminals that do not use the currently
: selected background color for erasing.  In my opinion, such terminals
: are brain-dead and there is really no way of properly optimizing the
: screen redrawing on these terminals.
: [...]
:
: For these reasons, I have decided not to support such poorly designed
: terminals with the hope that the terminal software itself will get
: fixed.  I modified rxvt to provide the bce capability and I believe
: that the new color xterms work this way as well.  Now someone needs to
: fix screen to make it bce-compliant.
: [...]

Regarding his stated role in the development of rxvt, I had noticed similar comments by Davis early on, and asked Mike Olesen about that in 1996, because I had also noticed that rxvt's change-log did not mention any involvement by Davis in the development of the “new color model” as shown here:

2.11 to 2.12
\-----------
...
    2.  New colour possibilities -- 16 colours.  Instead of using fatter
        characters, a bold attribute now uses a brighter foreground colour.
        Similarly, a blink attribute will use a brighter background colour.
        It is also possible to choose these colours through the resources
        (color0 -- color7) permit specification of the ANSI colours (black,
        red, green, yellow, blue, magenta, cyan, white), the resources
        (color10 -- color17) are the brighter bold/blink equivalents.
        Define USE_FAKE_BOLD to disable.
...
2.10 to 2.11
    1.  If NEW_COLOR_MODEL is defined in screen.c, the new model I described
        in an earlier email is used. (apparantly there are two, slightly
        differnt models for using color extensions in a vt-compatible
        terminal. This flag lets you switch.

Olesen replied that it was a lot of work (to make the colors work properly). Later Olesen worked with others to set up a mailing list rxvt-workers from which one might glean more information. But from Olesen's response I understood that while Davis might have provided initial changes, that Olesen did most of the actual work for bce in rxvt.

This was in 1995 (June to October or November). I have a copy of rxvt 2.11, and made changes to vile in November for rxvt 2.12 (which is no longer available).

I summarized this in the terminal database in October 2014:

# rxvt was originally "xvt", first announced in April 1993:
#       http://www.informatica.co.cr/linux-desktops/research/1993/0416.html
#
# Though its change-log does not mention this, John Davis has stated that he
# was the author of the changes to use the bce ("new color model") which was
# incorporated into rxvt 2.11 (June 15, 1995).  The change-log does not give
# dates, nor give developer's names.  Initial color support was added for rxvt
# "2.0", which was sometime in 1994.
#
# rxvt had usable color support with 2.16 (April 2, 1996), with some help by my
# work on vttest, as well as bug reports to Mark Olesen.  For instance, the fix
# mentioned here
#       http://web.archiveorange.com/archive/v/6ETvLb5wHtbbzCaS4S9J
# was from one of my bug-reports -TD
#
# While the color model both for xterm and rxvt was based on Linux console,
# Olesen (or possibly Davis) diverged in one respect from Linux's bce color
# behavior: inserting/deleting characters does not fill the newly empty cell
# with the default background color.
rxvt|rxvt terminal emulator (X Window System),

Others may disagree, crediting Davis with more than the changelogs show:

Re: Xterm compiled with cygnus gcc

    To: gnu-win32 at cygnus dot com
    Subject: Re: Xterm compiled with cygnus gcc
    From: vischne at ibm dot net
    Date: Wed, 19 Nov 1997 16:25:58 GMT

> However, Sergey has compiled "rxvt", a xterm look-alike that AFAIK
> only lacks the Tektronix support that xterm has. You should find

One comment: John Davis (davis@space.mit.edu,
ftp://space.mit.edu/pub/davis) used to maintain a private version of the
rxvt code that was much, much better than the public versions.  The last
time I tested a most-recent, public version of rxvt using Linux, it was
obvious that the Davis improvements hadn't reached the attention of the
implementor.

but over the years, I was (like this one) unable to find the ostensibly improved version:

Re: Xterm compiled with cygnus gcc

    To: vischne at ibm dot net
    Subject: Re: Xterm compiled with cygnus gcc
    From: Michael Anderson <mka at redes dot int dot com dot mx>
    Date: Thu, 20 Nov 1997 08:23:05 -0600
    CC: gnu-win32 at cygnus dot com
    References: <199711191625.QAA15790@out1.ibm.net>

vischne@ibm.net wrote:

> One comment: John Davis (davis@space.mit.edu,
> ftp://space.mit.edu/pub/davis) used to maintain a private version of the
> rxvt code that was much, much better than the public versions.

I looked at the above address and did not find the rxvt code. Do you
have another reference?

--

Mike Anderson
mka@redes.int.com.mx
Guanajuato, GTO, Mexico

Keeping that in mind, I replied on comp.editors in 2007 to a thread EPM workalike for Linux?:

Jamie <nos...@geniegate.com> wrote:

> Jed is a pretty darn good editor, it's very fast and easier to use (IMO) than
> vi. (written by the same J. Davis, who also brought us slrn and rxvt, so you
> know it was written extremely well)

John Davis contributed some fixes for rxvt, but by no means "brought us"
the program.

--
Thomas E. Dickey
http://invisible-island.net
ftp://invisible-island.net

Who did it?

Going back to Davis' question in September 1996 (whether to me or the world at large is unclear), Davis asked who came up with the color model for the Linux console.

One way to answer that question is to examine different early versions of the Linux kernel. For quite a while (i.e., until getting involved with BitKeeper after ten years of kernel development), that required comparing source-trees. While there was a mailing list, there was never a complete archive of that.

These provided useful information:

Color support for the Linux console came in 0.12 (January 16, 1992). Comparing that with 0.11 gives a clue:

Index: kernel/chr_drv/console.c
--- linux-0.11/kernel/chr_drv/console.c 1991-11-23 18:41:21.000000000 +0000
+++ linux-0.12/kernel/chr_drv/console.c 1992-01-12 20:28:33.000000000 +0000
@@ -13,6 +13,9 @@
  * Hopefully this will be a rather complete VT102 implementation.
  *
  * Beeping thanks to John T Kohl.
+ *
+ * Virtual Consoles, Screen Blanking, Screen Dumping, Color, Graphics
+ *   Chars, and VT100 enhancements by Peter MacDonald.
  */

/*

The source-code shows which SGR codes were implemented:

void csi_m(int currcons )
{
        int i;

        for (i=0;i<=npar;i++)
                switch (par[i]) {
                        case 0: attr=def_attr;break;  /* default */
                        case 1: attr=(iscolor?attr|0x08:attr|0x0f);break;  /* bold */
                        /*case 4: attr=attr|0x01;break;*/  /* underline */
                        case 4: /* bold */
                          if (!iscolor)
                            attr |= 0x01;
                          else
                          { /* check if forground == background */
                            if (vc_cons[currcons].vc_bold_attr != -1)
                              attr = (vc_cons[currcons].vc_bold_attr&0x0f)|(0xf0&(attr));
                            else
                            { short newattr = (attr&0xf0)|(0xf&(~attr));
                              attr = ((newattr&0xf)==((attr>>4)&0xf)?
                                (attr&0xf0)|(((attr&0xf)+1)%0xf):
                                newattr);
                            }
                          }
                          break;
                        case 5: attr=attr|0x80;break;  /* blinking */
                        case 7: attr=(attr<<4)|(attr>>4);break;  /* negative */
                        case 22: attr=attr&0xf7;break; /* not bold */
                        case 24: attr=attr&0xfe;break;  /* not underline */
                        case 25: attr=attr&0x7f;break;  /* not blinking */
                        case 27: attr=def_attr;break; /* positive image */
                        case 39: attr=(attr & 0xf0)|(def_attr & 0x0f); break;
                        case 49: attr=(attr & 0x0f)|(def_attr & 0xf0); break;
                        default:
                          if (!can_do_colour)
                            break;
                          iscolor = 1;
                          if ((par[i]>=30) && (par[i]<=38))
                            attr = (attr & 0xf0) | (par[i]-30);
                          else  /* Background color */
                            if ((par[i]>=40) && (par[i]<=48))
                              attr = (attr & 0x0f) | ((par[i]-40)<<4);
                            else
                                break;
                }
}

In the initial implementation, the video attribute was applied only to text. Other updates used the video_erase_char:

insert-character (ich), delete-character (dch), insert-line (il), delete-line (dl), scroll up/down, erase-display (ed), erase-line (el, el1).

The current value of attr could be saved using a control sequence provided for setterm, which in turn updated video_erase_char.

Someone (perhaps Mika Liljeberg as hinted in the release notes) decided to combine the two by adding update_attr in 0.96b (June 21, 1992):

/*
*  I hope this works. The monochrome part is untested.
*/
static void update_attr(int currcons)
{
        unsigned char a = color;

        if (can_do_color) {
                if (underline)
                        a = (a & 0xf8) | ulcolor;
                else if (intensity == 0)
                        a = (a & 0xf0) | halfcolor;
        }
        if (reverse ^ decscnm)
                a = (a & 0x88) | (((a >> 4) | (a << 4)) & 0x77);
        if (blink)
                a |= 0x80;
        if (intensity == 2)
                a |= 0x08;
        if (!can_do_color) {
                if (underline)
                        a = (a & 0xf8) | 0x01;
                else if (intensity == 0)
                        a = (a & 0xf0) | 0x08;
        }
        video_erase_char = (a << 8) | ' ';
}

and revising csi_m:

static void csi_m(int currcons)
{
        int i;

        for (i=0;i<=npar;i++)
                switch (par[i]) {
                        case 0: /* all attributes off */
                                intensity = 1;
                                underline = 0;
                                reverse = 0;
                                blink = 0;
                                color = def_color;
                                break;
                        case 1:
                                intensity = 2;
                                break;
                        case 2:
                                intensity = 0;
                                break;
                        case 4:
                                underline = 1;
                                break;
                        case 5:
                                blink = 1;
                                break;
                        case 7:
                                reverse = 1;
                                break;
                        case 21:
                        case 22:
                                intensity = 1;
                                break;
                        case 24:
                                underline = 0;
                                break;
                        case 25:
                                blink = 0;
                                break;
                        case 27:
                                reverse = 0;
                                break;
                        case 39:
                                color = (def_color & 0x0f) | background;
                                break;
                        case 49:
                                color = (def_color & 0xf0) | foreground;
                                break;
                        default:
                                if (par[i] >= 30 && par[i] <= 37)
                                        color = color_table[par[i]-30]
                                                | background;
                                else if (par[i] >= 40 && par[i] <= 47)
                                        color = (color_table[par[i]-40]<<4)
                                                | foreground;
                                break;
                }
        update_attr(currcons);
}

The changes to console.c were the largest for that patch, but not so large that it would have likely been a collaboration.

Linus' announcement for 0.96b did not mention any change to color behavior, although it mentioned other changes to the console driver (VT100s don't do color):

commit 0b098c16bd53bb586673c76b0eb137e0b09f4f5f (Jun 21 1992)
Author: Linus Torvalds <torvalds@linuxfoundation.org>
Date:   Fri Nov 23 15:09:01 2007 -0500

    [PATCH] Linux-0.96b (June 21, 1992)

    First cut at core-dumping.  Also, do more dynamic boottime memory
    allocation, rather than allocating data structures statically.

    Allocate task structures at run-time rather than having a big array of
    them.

    First "obsolete" system call.  The old "stat()" system call was too
    limited, due to "struct stat" having various 16-bit fields etc (notably
    inode numbers).  We make a new stat() function, and keep the old one
    around as "old_stat()" for binary compatibility.

    We also need a bigger "utsname" to hold real node names.

    Whoo! NR_OPEN is now 32 rather than 20.

    itimer() support driven by X11 adoption (Darren Senn).

    gcc starts using fsqrt, so that gets added to the FP emulation.  We're
    still basing that on my trivial code.

    [Original changelog below]

    0.96b is not a new major release: it's pretty close to 0.96a with all my
    patches (1-4).  However, as there has been 4 patches already, I decided
    it would be time for a full kernel release along with a bootimage, so
    that people who don't feel confident with patching can use the new
    features.

    If you already have 0.96a patchlevel 4, 0.96b will offer you these new
    features:

     - the math-emulation now handles fsqrt, as gcc-2.2.2 generates that
       inline.  I haven't tested the kernel code at all: I tested the
       algorithm in user space, but I'm lazy, so I never turned off my 387
       to do real testing.  I hope it works.
     - better vt100 terminal emulation thanks to Mika Liljeberg.
     - I removed a possible race-condition in the buffer-cache code.
     - minor fixes

    The vt100 emulation should now be complete enough for almost everything
    (including vt100 test suites): as a result the setterm utility had to be
    changed (as the old setterm codes aren't compatible with the full vt100
    codes).  setterm-0.96b.tar.Z contains the new setterm.

    The soon-to-be-released gcc-2.2.2 will need the 0.96b kernel: (a) due to
    the fsqrt emulation and (b) it uses the new stat() system call.  So
    upgrading is a good idea.  (If you have a co-processor, (a) isn't used,
    but (b) still stands)

    If you have an unpatched 0.96a, the differences to 0.96b are roughly
    (not counting the above-mentioned new things):

     - corrected the disk-buffer-list bug with read/write-errors
     - fixed read-ahead warning messages at end of disk
     - better support for text-mode restoration after running MGR and X
     - full core-dumping, attach/detach etc debugging features
     - 16550A support
     - less low 1MB memory used for kernel structures
     - various minor fixes

    Note that the fact that new versions (pl4 and above) use more memory in
    the 1M+ area means that linux will report less free memory (it's used
    for buffer-cache instead).  This could concievably be a problem on 2MB
    machines.  The standard kernel comes with only 4 pty's though, and if
    you use the standard 80x25 text modes instead of svga modes, the VC
    buffers will be smaller.  Please contact me if there are problems even
    with this minimal setup.

    0.96b does /not/ contain: the new scsi drivers, new filesystems or some
    other patches I have gotten (ibm character set mode, loop-devices etc).
    If you have sent me any other patch, you might want to remind me about
    it.

Linus

Aside from mailing-lists, the documentation as such lagged. Linux 1.00 in March 1994 added a brief CHANGES and a CREDITS file. That overlooked Peter MacDonald until January 1995:

diff -u --recursive --new-file v1.1.77/linux/CREDITS linux/CREDITS
--- v1.1.77/linux/CREDITS       Sun Jan  1 19:49:18 1995
+++ linux/CREDITS       Mon Jan  9 06:40:59 1995
@@ -478,6 +478,10 @@
 E: Kai.Makisara@vtt.fi
 D: SCSI Tape Driver

+N: Peter MacDonald
+D: SLS distribution
+D: Initial implementation of VC's, pty's and select()
+
 N: James B. MacLean
 E: jmaclean@fox.nstn.ns.ca
 D: Coordinator of DOSEMU releases

Oddly enough the style and terseness of the initial CHANGES looked like something that John E. Davis would write for slang. Here is the beginning (the file is 143 lines):

CHANGES since 0.99 patchlevel 15:

 - removed all the bugs, of course.
 - networking fixes.
 - more changes than I really wanted..

CHANGES since 0.99 patchlevel 14:

 - too many to count, really.  Besides, I've lost my notes.

CHANGES since 0.99 patchlevel 13:

 - new kernel source layout: drivers separated
 - lots of networking bugs fixed, and new network card drivers (Alan Cox,
   Donald Becker &co)
 - sound driver added to the default source distribution (Hannu
   Savolainen)
 - updated SCSI driver code (Eric Youngdale, Drew Eckhardt &co)
 - readonly OS/2 filesystem support (HPFS) added (Chris Smith)
 - NTP support (Philip Gladstone, Torsten Duwe, ??)
 - fixed 16MB swap-area limit
 - lots of minor cleanups, buxfixes etc.

CHANGES since 0.99 patchlevel 12 and earlier:

 - the bad memory management one-liner bug in pl12 is naturally fixed.
 - compiled with plain C by default instead of C++
 - ELF binary support (Eric Youngdale)

I started developing with Linux in March 1994. Based on advice from Guy Cox, who had been using Linux since the previous fall, I went looking for Peter MacDonald's SLS (see for example SLS update: getty, sysvinit, and emacs X11 6/1/93), and found Slackware instead (see Interview with Patrick Volkerding, Apr 01, 1994 By Phil Hughes). Slackware used the “new” 0.99-15 kernel.

With minor refinements, the Linux console color behavior was complete. For instance, erase-character (ech) came later. That is a VT220 feature (in itself not much, but a full VT220 emulation would have more than doubled the size of the console driver). Since it was an “erase” it inherited the behavior of other erase-features. Oddly, this was overlooked in rxvt when its developers added the same feature. The terminal description for Linux console uses ech while rxvt does not.

One aspect which surprises users is that the currently-selected colors are applied to the blank area which is scrolled in, e.g., by a new line at the bottom of the screen. Occasionally someone tries to “fix” this without understanding how it all works together. In 2008, an unwary committer accepted a patch for the Linux console to change the behavior. After some discussion (and bug reports), it was reverted :

Commit 93f78da4
authored 8 years ago by Linus Torvalds's avatar Linus Torvalds

Revert "vt: fix background color on line feed"

This reverts commit c9e587ab, and the
subsequent commits that fixed it up:

 - afa9b649 "fbcon: prevent cursor disappearance after switching to 512
   character font"

 - d850a2fa "vt/fbcon: fix background color on line feed"

 - 7fe3915a "vt/fbcon: update scrl_erase_char after 256/512-glyph font
   switch"

by request of Alan Cox. Quoth Alan:
  "Unfortunately it's wrong and its been causing breakages because
   various apps like ncurses expect our previous (and correct)
   behaviour."

Alexander sent out a similar patch.

Requested-by: Alan Cox <alan@lxorguk.ukuu.org.uk>
Tested-by: Jan Engelhardt <jengelh@medozas.de>
Cc: Alexander V. Lukyanov <lav@netis.ru>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Notwithstanding all of that, users still are startled, and developers attempt to “fix” the problem, but only succeed in making new bugs, as illustrated by SGR reset code is ignored if \n is placed on a new line.

Making it work

I began work on xterm early in January 1996, outlining the work that I planned to improve color handling, as well as other fixes (such as compiler warnings). These points from my email to Dawes and Wexelblat are relevant:

        + modify the operation of the ClearRight, ClearBufRows functions so
          that (when BG_COLOR is set!) background color propagates to the
          cleared cells.

          In my version I juggled the GC's in the CASE_SGR in charproc.c; This
          is the point that I have to review to determine if it's necessary (to
          make ClearRight, ClearBufRows work properly).

          => this is the primary change needed to allow use of fullscreen
             applications (minicom, jed, vile).

             While testing this, I noticed that ncurses applications
             (incorrectly) are filling every cell with color, rather than using
             ED, EL (_that_ will change, since that's one of the reasons I've
             been working on ncurses).

        + correct the way background color is used, so that the application can
          set a white background.

That is, I planned to implement bce in xterm.

Starting two months later (March 5), I participated in the review for X/Open Curses Issue 4 Version 2. I made several comments on the mailing list (some useful, since I was able to point out some errors and inconsistencies in the text). Eric Raymond joined the mailing list a week later, but apparently had nothing to say. The review ended three months later, leaving me unsatisfied (most of the proposed changes were minor).

I was interested in the review because I had thought it would provide a way to propose extensions to the documented X/Open Curses. I asked about that, but the response was not exactly helpful:

From ajosey@xopen.co.uk Fri Mar 15 10:56 EST 1996
Received: from xopen.xopen.co.uk (xopen.xopen.co.uk [192.153.166.4]) by mail.Clark.Net (8.7.3/8.6.5) with SMTP id KAA04221 for <dickey@clark.net>; Fri, 15 Mar 1996 10:56:17 -0500 (EST)
Received: by xopen.xopen.co.uk; id AA04003; Fri, 15 Mar 1996 15:57:47 GMT
Received: by xopuk.xopen.co.uk
        (1.36.108.3/16.2) id AA19341; Fri, 15 Mar 96 15:52:15 GMT
Message-Id: <9603151557.AA04003@xopen.xopen.co.uk>
From: ajosey@xopen.co.uk (Andrew Josey)
Date: Fri, 15 Mar 1996 15:52:14 +0000
In-Reply-To: "T.E.Dickey"'s message as of Mar 15, 10:41am.
Reply-To: ajosey@xopen.org (Andrew Josey)
X-Mailer: Mail User's Shell (7.2.5 10/14/92)
To: "T.E.Dickey" <dickey@clark.net>
Subject: Re: (XoCurses 10) Re: (XoCurses 9) Re: (XoCurses 8) Re: (XoCurses 7) Re: (XoCurses 4) comments on draft changes
Content-Type: text
Content-Length: 457
Status: RO

hi Thomas

By all means you can propose them. However X/Open we are
presently concentrating on trying to get the specification complete
and correct as is.

> After review:
> what about extensions?  I've got two that I'd like to see incorporated:
>
>       resizing windows (and the whole screen)
>
>       incorporate support for ISO-6429 _default_ colors
>
> (both of these are tied in with an application I've been working on for
> some time).

regards
Andrew

More to the point, I had no existing implementation to refer to, and thought that was a problem.

Writing more than 20 years later, there is an activity going on to ”standardize” SIGWINCH (mostly preserving existing practice, but with some changes which will impact existing applications). In the interim, I also participated in the review for X/Open Curses Issue 7, in 2009. Again, the purpose of the review was to comment on minor errata overlooked in 1996. Along with 3 dozen other issues, I pointed out a problem with tparm, proposing a <stdarg.h> implementation tiparm which was incorporated into the document. While none of the Unix vendors have updated their curses implementations since the mid-1990s, NetBSD added this feature in 2011.

Standardization, of course, should reflect existing practice. When the documented behavior lags noticeably, it is not really standardization.

Around the same time (March/April 1996) I started talking to Mike Olesen, who was the main developer working on rxvt. We discussed special keys (cursor-, keypad-keys) and color. I outlined my goals for making a 9th color for ncurses applications, the “default” color.

Later (if you read my discussion of ncurses, you may understand why it was slow), in November 1996 I discussed the default colors with John Davis.

We agreed it was a good idea, then immediately disagreed on how to provide it to applications. Davis proposed adding an environment variable (COLORFGBG), while I saw drawbacks to that approach, and used a function. Bear in mind that I wanted to use the feature in my directory editor automatically, based on the terminal's capabilities.

I was not interested in carrying on the discussion in the newsgroup:

From davis@space.mit.edu  Sun Nov  3 14:21:22 1996
Received: from space.mit.edu (SPACE.MIT.EDU [18.75.0.10]) by mail.clark.net (8.7.3/8.6.5) with SMTP id OAA10413 for <dickey@clark.net>; Sun, 3 Nov 1996 14:21:21 -0500 (EST)
Received: from aluche.mit.edu by space.mit.edu AA13469; Sun, 3 Nov 96 14:21:14 EST
From: "John E. Davis" <davis@space.mit.edu>
Received: (from davis@localhost) by aluche.mit.edu (8.7/8.7) id OAA13423; Sun, 3 Nov 1996 14:21:14 -0500
Date: Sun, 3 Nov 1996 14:21:14 -0500
Message-Id: <199611031921.OAA13423@aluche.mit.edu>
To: dickey@clark.net
Posted-To: comp.os.linux.development.apps
Subject: Re: Changing terminal's *normal* color scheme?"
References: <slrn57n9gc.b4p.louis@lcjones.aclib.siue.edu> <55i231$s6r@clarknet.clark.net>
Organization: Center for Space Research
Reply-To: davis@space.mit.edu
Status: RO

[I have also posted the following text to Usenet.]

On 3 Nov 1996 12:10:09 GMT, T.E.Dickey <dickey@clark.net>
wrote:
>there's no _standard_ solution.  XSI (SVr4) curses applications "assume" that
>the screen is white-on-black.  So when they're running they paint the screen
>appropriately.  When exiting, they use the terminal's reset capabilities.
>_Some_ terminals can reset the terminal to the "default" using SGR 39, SGR 49
>(what you can set/save with setterm on the Linux console).  But this isn't
>portable (most terminals don't have a save-colors control).

I am very familiar with this problem.  The next version of slang
(0.99-36) will look for an environment variable called COLORFGBG whose
value is assumed to consist of a foreground color name and a
background color name separated by a semi-colon, e.g.,

    setenv COLORFGBG "green;black"

I am not sure how well this solution will work in practice.
--
John E. Davis                   Center for Space Research/AXAF Science Center
617-258-8119                    MIT 37-662c, Cambridge, MA 02139
http://space.mit.edu/~davis

and followed up with a comment about my plan to implement the feature in ncurses. Davis added a few details:

From davis@space.mit.edu  Mon Nov  4 19:35:38 1996
Received: from space.mit.edu (SPACE.MIT.EDU [18.75.0.10]) by mail.clark.net (8.7.3/8.6.5) with SMTP id TAA23975 for <dickey@clark.net>; Mon, 4 Nov 1996 19:35:37 -0500 (EST)
Message-Id: <199611050035.TAA23975@mail.clark.net>
Received: from wiwaxia.mit.edu by space.mit.edu AA15800; Mon, 4 Nov 96 19:35:34 EST
Received: by wiwaxia AA19771; Mon, 4 Nov 96 19:35:33 EST
Date: Mon, 4 Nov 96 19:35:33 EST
From: "John E. Davis" <davis@space.mit.edu>
To: dickey@clark.net
Subject: Re: Changing terminal's *normal* color scheme?"
X-Mailer: Jed [0.98-0]
Status: RO

>>     setenv COLORFGBG "green;black"
>>
>> I am not sure how well this solution will work in practice.
>I've had on my list (since last year) to extend ncurses so it "knows" that
>the original color pair can be restored.  How 'bout something a little more
>mnemonic such as DEFAULT_COLORS "green;black"?  (It _is_ the "default"
>or "original" colors that you'd like to make the application transparent to).

I will support both.  The problem now would be to standardize the
names of the colors.  The names that slang uses are in this table:

   {"black",            SLSMG_COLOR_BLACK},
   {"red",              SLSMG_COLOR_RED},
   {"green",            SLSMG_COLOR_GREEN},
   {"brown",            SLSMG_COLOR_BROWN},
   {"blue",             SLSMG_COLOR_BLUE},
   {"magenta",          SLSMG_COLOR_MAGENTA},
   {"cyan",             SLSMG_COLOR_CYAN},
   {"lightgray",        SLSMG_COLOR_LGRAY},
   {"gray",             SLSMG_COLOR_GRAY},
   {"brightred",        SLSMG_COLOR_BRIGHT_RED},
   {"brightgreen",      SLSMG_COLOR_BRIGHT_GREEN},
   {"yellow",           SLSMG_COLOR_BRIGHT_BROWN},
   {"brightblue",       SLSMG_COLOR_BRIGHT_BLUE},
   {"brightmagenta",    SLSMG_COLOR_BRIGHT_CYAN},
   {"brightcyan",       SLSMG_COLOR_BRIGHT_MAGENTA},
   {"white",            SLSMG_COLOR_BRIGHT_WHITE}

--John

Interestingly enough, although Davis added “default” (no “original”):

#define MAX_COLOR_NAMES 17
static Color_Def_Type Color_Defs [MAX_COLOR_NAMES] =
{
     {"black",          SLSMG_COLOR_BLACK},
     {"red",            SLSMG_COLOR_RED},
     {"green",          SLSMG_COLOR_GREEN},
     {"brown",          SLSMG_COLOR_BROWN},
     {"blue",           SLSMG_COLOR_BLUE},
     {"magenta",        SLSMG_COLOR_MAGENTA},
     {"cyan",           SLSMG_COLOR_CYAN},
     {"lightgray",      SLSMG_COLOR_LGRAY},
     {"gray",           SLSMG_COLOR_GRAY},
     {"brightred",      SLSMG_COLOR_BRIGHT_RED},
     {"brightgreen",    SLSMG_COLOR_BRIGHT_GREEN},
     {"yellow",         SLSMG_COLOR_BRIGHT_BROWN},
     {"brightblue",     SLSMG_COLOR_BRIGHT_BLUE},
     {"brightmagenta",  SLSMG_COLOR_BRIGHT_MAGENTA},
     {"brightcyan",     SLSMG_COLOR_BRIGHT_CYAN},
     {"white",          SLSMG_COLOR_BRIGHT_WHITE},
     {"default",        SLSMG_COLOR_DEFAULT}
};

he did not add the symbol to slang.h which would facilitate programs using the feature. That is an internal detail in sldisply.c, initially the magic number 25, but as of 2.3.1a, a conditionally-defined value. His comment about adding both refers to the DEFAULT_COLORS environment variable.

While it seemed “obvious” what design choices to make, I found that there were quirks in the bce behavior in other implementations of curses:

From dickey Sun Nov 30 07:21:29 1997
Subject: problem with curses+color
To: urs@akk.uni-karlsruhe.de (Urs Janßen)
Date: Sun, 30 Nov 1997 07:21:29 -0500 (EST)
Cc: tin-dev@tin.org (Unofficial TIN)
X-Mailer: ELM [version 2.4 PL24alpha3]
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
Status: RO
Content-Length: 1346
Lines: 25

Now that things are getting more stable, I built another copy of tin
yesterday with Solaris curses, and found problems - perhaps it's a bug
introduced in the 2.5.1 curses.  Unlike ncurses, it seems that the Solaris
curses assumes that erase-display _and_ erase-line both reset colors (I can
understand the first, since it's an issue that I've been researching).
However, minicom on Linux console and the XFree86 xterm assume that it
doesn't.  That makes the Solaris version keep the blue-bar on more places
on the screen than it should.  (For some other reason, I'm not having
troubles with dropped characters; I'd thought that wasn't fixed until 2.6,
which was why I wasn't paying a lot of attention to it recently).

Anyway:  the result is that tin works fine with ncurses in color, but not
for the emulators I'm using.  I'm contemplating adding a resource to xterm
to match the other color behavior since I understand that the vt525 does
this, i.e., resets color when an erase-display is done.  When I do that,
I'll try testing it with tin to see if that can iron out things.  However,
it won't "fix" minicom, so there's the issue that running tin built with
Solaris curses via minicom to a Linux console won't work well.

(The curses build does perform better, though).

--
Thomas E. Dickey
dickey@clark.net
http://www.clark.net/pub/dickey

The reason for a difference in behavior is that one or more of the low-level operations that might be related to bce differed between the assumptions in Solaris curses and ncurses. A likely place to look (based on reading the Solaris source code in Illumos) might be the scrolling operations which use insert/delete line control sequences.

While differences in library capability may have been surprising, I was already aware that there were differences in the terminal behavior:

970614
        + remove ech capability from rxvt description because it does not work.

although Davis seems to have overlooked that because slang did not use the corresponding ec capability.

It was noticeable in ncurses because xterm (like the Linux console) fills with the current color on all of those low-level operations, and because ncurses chooses from the most efficient of each of those operations. If a terminal does not fill with the current color for one of those, then the fix (for ncurses) is to omit the deviant capability from the terminal description. Later, in 2004, I added a screen to vttest to illustrate this difference, having found that the earlier test-screen was not enough:

20040804
        + change parameter for ioctl(fd, FIONREAD, &param) to an int.

        + add a test screen for BCE color behavior with ECH, DCH and scrolling.

Here are screenshots illustrating the problem with rxvt (with xterm on the left):

Most of the discussion regarding bce in slang versus ncurses dwelt upon slang's lack of support for terminals which did not provide this feature.

In 1999, Davis commented that he had modified slang to work with terminals that did not use the bce feature:

From dickey Thu May  6 05:18:43 1999
From: "T.E.Dickey" <dickey@shell.clark.net>
Subject: Re: 'screen' and dselect/lynx/mutt/slrn (terminfo?)
Newsgroups: comp.os.linux.misc,comp.terminals
References: <slrn7idjui.vi4.timsuth@fragolian.zone> <IX3W2.230$qh4.14590@iad-read.news.verio.net> <slrn7iioqc.kf.timsuth@fragolian.zone> <slrn7irqpk.3ac.davis@mygir.davis.net> <7gpgn4$s1h$1@newsmaster.cc.columbia.edu> <slrn7j2ksi.8ba.davis@aluche.mit.edu>
Organization: Clark Internet Services, Inc., Ellicott City, MD USA
User-Agent: tin/pre-1.4-19990413 ("Endemoniada") (UNIX) (SunOS/5.6 (sun4u))
Status: RO
Content-Length: 530
Lines: 14

In comp.os.linux.misc John E. Davis <davis@space.mit.edu> wrote:

> My development version of slang handles terminals that do not have the
> bce capability, and it was not difficult to implement.  I was
> reluctant to add support for such terminals because I do not want to
> encourage their use.

well, as long as your idea of how non-bce is implemented matches the
assumptions in existing software, you won't be introducing any new
problems by releasing it.

--
Thomas E. Dickey
dickey@clark.net
http://www.clark.net/pub/dickey

I investigated, noticed some problems as I reported in the same newsgroup:

From dickey Mon Jul 26 08:52:26 1999
From: "T.E.Dickey" <dickey@shell.clark.net>
Subject: Re: Lynx and s-lang 1.3.8
Newsgroups: alt.lang.s-lang
References: <7ncnhg$j2v$1@news.pubnix.net>
Organization: Clark Internet Services, Inc., Ellicott City, MD USA
User-Agent: tin/pre-1.4-19990624 ("Dawnrazor") (UNIX) (SunOS/5.6 (sun4u))
Status: RO
Content-Length: 1497
Lines: 27

Ismael Cordeiro <ismael@pubnix.net> wrote:
> For about two years I've been compiling lynx with s-lang on Solaris2 and
> FreeBSD (presently Solaris2.5 and FreeBSD 2.2.7). I've never had a problem
> and I've always had better results with s-lang than with Solaris' curses or
> ncurses. With s-lang 1.3.8 I got a problem for the first time. The same
> version of lynx compiled with s-lang 1.3.7 and 1.3.8 give different results.
> The one compiled with 1.3.8 gives me some garbage on the screen on the
> FreeBSD system. It seems that the problem is related to how the screen is
> cleared. I've always used the "ut" capability (screen erased with background
> color) in my termcap on the FreeBSD system ("bce" in Solaris' terminfo). When
> I remove "ut" from my termcap the problem with s-lang 1.3.8 disappears and I
> get no garbage on the screen. On the Solaris 2.5 system I have no problems.
...
> Any ideas about what is happening?

John's added partial support for bce.  I haven't looked very closely except
to see that it doesn't work as expected (he said he'll look into it when he
comes back from vacation -- iirc next week).

If you were using 'ut' in your termcap with slang, that's an error anyway,
since slang hasn't supported it.  (If you want to see it working properly,
look at ncurses - the differences between ncurses implementation of bce and
Solaris' are small enough that I'm probably the only one who sees ;-)

--
Thomas E. Dickey
dickey@clark.net
http://www.clark.net/pub/dickey

Revisiting this issue in 2017, I still see problems with slang's implementation: SLcurses applications are unimproved, although Davis has provided a demo in the smgtest program to show that non-bce terminals “work” (in contrast to the ncurses test-programs which do not require a special screen for illustrating the behavior).

The change alluded to by Davis would have to be between 1.2.2 (April 1998) and 1.3.6 (May 1999). The changes.txt file does not mention anything that is relevant. The source-code is the documentation.

Here is the relevant code from smgtest.c:

static void post_test (void)
{
   write_centered_string ("Press any key to return.", SLtt_Screen_Rows - 1);
   SLsmg_refresh ();
   SLsig_unblock_signals ();
   (void) SLkp_getkey ();
}

static void bce_color_test (void)
{
   int row;

   if (-1 == check_color_support ())
     return;

   pre_test ("Background Color Erase Test");

   SLtt_set_color (0, NULL, "lightgray", "blue");

   row = SLtt_Screen_Rows/2;
   SLsmg_set_color (0);
   SLsmg_gotorc (row, 1);
   SLsmg_write_string ("The screen background should be blue.");

   row += 2;
   SLsmg_gotorc (row++, 1);
   SLsmg_write_string ("****If the screen update is slow, then your terminal");
   SLsmg_gotorc (row++, 1);
   SLsmg_write_string ("    does not support background-color-erase.");
   SLsmg_set_color (0);
   post_test ();
}

That is, it writes a couple of lines of text on a blank screen and waits for the user to press Enter. Among other things, it does not:

scroll the screen up/down
switch between different non-default colors
erase parts of the screen

One could make a rather long list of things to test; smgtest.c is rather short.

Here are screenshots to demonstrate the test, comparing xterm and tmux xterm on the left):

The portions of the top/bottom lines using the terminal's default background color when using tmux indicate a problem with the test results. I probably noticed something like that when I ran the test program in 1999. These screenshots use the latest version of slang.

Here are screenshots with tmux using vttest, to demonstrate that it is one of the terminal (emulators) which do not support bce:

The limitations of slang's support for non-bce terminals is more apparent when viewing a non-trivial test program. Here are screenshots comparing ncurses and slang for the hanoi program:

Part of the problem is hinted at in this item from Lynx's changes:

2000-06-02 (2.8.4dev.3)
...
* for slang configurations past 1.3.6, call SLsmg_touch_screen() in
  start_curses to ensure that the screen is properly initialized for non-bce
  terminals -JED (John E Davis)

That is, Davis left an impediment in slang which discourages people from using screen, tmux and similar programs. Bear in mind that Lynx does not use SLcurses, but uses Davis' preferred library interfaces.

SLsmg_touch_screen, added after 1.2.2, is undocumented. It is mentioned in changes.txt:

61. Changes made to SLtt/SLsmg code so that when a color definition is
    made via, e.g., SLtt_set_color, then the SLsmg interface will be
    get automatcally notified so that the next SLsmg_refresh will
    produce the correct colors.  In addition, SLsmg_touch_screen added.

The similarly-named SLsmg_touch_lines is documented with a caveat:

      Notes
        This function should rarely be called, if ever. If you find that
        you need to call this function, then your application should be
        modified to properly use the SLsmg screen management routines.
        This function is provided only for curses compatibility.

The two functions look different:

void SLsmg_touch_lines (int row, unsigned int n)
{
   int i;
   int r1, r2;

   /* Allow this function to be called even when we are not initialied.
    * Calling this function is useful after calling SLtt_set_color
    * to force the display to be redrawn
    */

   if (Smg_Mode == SMG_MODE_NONE)
     return;

   if (0 == compute_clip (row, (int) n, Start_Row, Start_Row + Screen_Rows, &r1, &r2))
     return;

   r1 -= Start_Row;
   r2 -= Start_Row;
   for (i = r1; i < r2; i++)
     {
        SL_Screen[i].flags |= TRASHED;
     }
}

void SLsmg_touch_screen (void)
{
   Screen_Trashed = 1;
}

But they have much the same effect, when you see how Screen_Trashed is used:

void SLsmg_refresh (void)
{
   unsigned int i;
#ifndef IBMPC_SYSTEM
   int trashed = 0;
#endif
   int r, c;

   if (Smg_Mode == SMG_MODE_NONE) return;

   if (Screen_Trashed)
     {
        Cls_Flag = 1;
        for (i = 0; i < Screen_Rows; i++)
          SL_Screen[i].flags |= TRASHED;
#ifdef REQUIRES_NON_BCE_SUPPORT
        adjust_colors ();
#endif
     }

The two functions could have been better integrated. If they had, the change to Lynx would not have been needed.

After a while Davis reduced the comments about non-bce terminals, as seen in this reply on news.software.readers in the thread slrn 0.9.7.2 color problem:

On 1 Oct 2001 03:07:16 GMT, E. <eri...@sw-tech.com>

>I'm using 1.2.2.  I've made 1.4.4 in my home dir, but can't get slrn to
>use it instead of the system ones.  I'm guessing this is due to my lack
>of familiarity with Linux, since I've had no problems doing so under
>multiple BSD flavors.

The problem that you are experiencing is due to lack of BCE
(background color erase) support in slang 1.2.x.  Upgrading to 1.4.x
will solve that problem.  Of course you should consider using a
terminal that implements BCE.

Compiling slang+slrn should be quite trival:

     1.  Put both slang and slrn tar files in the same directory,
         e.g., ~/src

     2.  cd ~/src
     3.  tar xf slang-1.4.4.tar
     4.  ln -s slang-1.4.4 slang
     5.  cd slang
     6.  ./configure; make
     7.  cd ../
     8.  tar xf slrn-xxx.tar
     9.  cd slrn-xxx
    10.  ./configure
    11.  make

The slrn configure script should find libslang.a in slang/src/objs.
This is why the symbolic link was created in step 4.

--John

Not all issues with screen and tmux are related to bce, but originate in slang's (related) use of hard-coded tests to determine whether a terminal matches Davis's assumptions.

For example, a thread on slang-users in 2016, [slang-users] Re: Treat screen* and tmux* as almost_vtxxx points to a problem with the way slang handles SIGWINCH. That was addressed in a Midnight Commander bug report: Subshell output lost on window resize under tmux, GNU screen

Use of color

Curses color pairs are separate from video attributes such as bold and underline. The terminfo capability ncv (no_color_video) lists video attributes that cannot be combined with color. For instance, a terminal may represent underlining using a particular color. SVr4 curses and ncurses check this information to ensure that color is used even if the video attribute cannot be shown at the same time.

S-Lang's color objects include some video attributes (bold, underline and reverse) as well as color. In applications (Lynx, Mutt, Midnight Commander), slang treats those video attributes as if they were just varieties of color. Here is a comment and reply on that issue:

Date: Thu, 10 Apr 1997 06:00:40 -0500 (CDT)
From: Klaus Weide <kweide@tezcat.com>
To: "T.E.Dickey" <dickey@clark.net>
Subject: Re: progress...

On Thu, 10 Apr 1997, T.E.Dickey wrote:

> > > I'll see how to adjust my ncurses patch to match that
> > > behavior.
> > 
> > Just don't adjust it where it doesn't make sense anyway...
> That's what I'm saying - the slang interface overloads colors 4 & 5 with
> underlining; so I'll look into doing that (to the "right" color pairs) in
> ncurses.

Well that overloading never made sense to me... at least not for the linux
console in color where physical underlining is not available.  I asked
about the reason for it a long time ago, nobody seemed to know...
I actually find it *nice* of your curses color code that it currently
*doesn't* do that.  Since underlining is translated into a color pair
anyway, why send A_UNDERLINE in addition?  Only makes things harder
to understand...  Without it, I can actually get the colors I have set
in lynx.cfg.  (modulo flipping, on a bad day.)

   Klaus

Perhaps that approach was due to the influence of MS-DOS on Davis' thinking, back in 1994. The MS-DOS influence shows clearly in the handling of other common video attributes:

If the terminal description lacks the standout feature, and it has reverse, then slang treats standout as if it had been defined to the same value as reverse.

This may sound reasonable, but it differs from BSD curses (which used underline as an alternative), and SVr4 curses (which assumed that standout was a feature of sgr). A terminal emulator using MS-DOS would be more likely to support reverse than underline, since the latter was a monochrome feature.
From the outset, when Davis began developing the screen management functions with Linux (seen in 0.99-7), there was a comment about blink:

        if (fgbg & SLTT_BLINK_MASK)
          {
             /* Someday Linux will have a blink mode that set high intensity
              * background.  Lets be prepared.
              */
             if (SLtt_Blink_Mode) SLtt_write_string (Blink_Vid_Str);
          }

That is, he intended the blink mode to be used for backgrounds as it might with ansi.sys to provide the same effect as bold applied to text. After all, Linux console implemented bold using the video driver's bright colors, so why not use the same device-specific feature for backgrounds?

S-Lang's MS-DOS driver uses the feature. As used in Lynx, the driver gives black text on a white background, which some users prefer to PDCurses' black-on-gray.

The SLtt_Blink_Mode variable was initialized to 1 for about ten years, until the release of 2.0.0 in April 2005. Since then, it has been initialized to 0, making the feature inactive by default. But the variable is still there, just in case.

Along the same lines, this hard-coded feature was added in 0.99-30, again attempting to use blink in an unexpected way:

   /* Although xterm cannot blink, it does display the blinking characters
    * as bold ones.  Some Rxvt will display the background as high intensity.
    */
   if ((NULL == (Blink_Vid_Str = tt_tgetstr("mb")))
       && is_xterm)
     Blink_Vid_Str = "\033[5m";

and is still present in 2017 (2.3.1a), although I implemented blinking text in xterm in 2004 (patch #185).

In the mail quoted, Klaus and I were discussing the problems in integrating Robert Partington's “color-style” feature into Lynx. It assigns a hash-code to HTML tags, and colors the content. That gives far more information about a webpage than the original scheme, which Davis and Macrides developed the year before. In retrospect it would have been technically feasible to make that work with slang:

We had a working example of how to manage colors with slang, and
color objects are not really different from color pairs.

But the absence of documentation and (apparent) disinterest on the part of Davis resulted in a curses-only implementation.

After several years (for Lynx 2.8.6, in 2006) I changed the configuration script default (for curses libraries) to use the color-style. To address compatibility issues, I added a script (oldlynx) and improved the color-style configuration to allow simulating the original color scheme.

Here are a couple of screenshots using the color-style configuration showing different color schemes:

and here are screenshots showing the original color scheme (left) and simulated original scheme (right):

Environment variables

In the discussion of Color Model, there is a difference in approach between Davis and me:

I am concerned with compatibility between ncurses and other implementations of curses. Some features (which should be controlled by the application) are provided as functions which the application can manipulate.
Davis does not have those concerns, is usually interested in a single application, and adds (or modifies) an environment variable which the end user can set.

None of this is new; see for example the mailing list thread in 2000 on mutt-dev about Color problems. Davis' position is expressed thus:

These variables do not replace any part of terminfo. S-Lang will work just fine without these variables provided that one has a properly configured terminfo file. However, it has been my experience that such files are rare. These variables were added to slang in order to give the user a convenient mechanism to make up for inadequate terminfo files. In fact, I think that anyone who has had to define COLORTERM in order to get colors can appreciate the usefulness of these environment variables.

In other words, none of the available terminal descriptions (in particular, given the time and audience, those from ncurses) match his hard-coded assumptions.

The environment variables recognized by ncurses are documented in the manual page. These are the environment variables recognized by slang:

COLORFGBG

Added in 0.99-36, this is nonstandard. It is documented (compare with assume_default_colors):

Unfortunately there does not seem to be a standard way for the application or, in particular, the library to determine which color will be used by the terminal for the default background. Such information would be useful in initializing the foreground and background colors associated with the default color object (0). For this reason, it is up to the application to provide some means for the user to indicate what these colors are for the particular terminal setup. To facilitate this, the SLtt_get_terminfo function checks for the existence of the COLORFGBG environment variable. If this variable exists, its value will be used to initialize the colors associated with the default color object. Specifically, the value is assumed to consist of a foreground color name and a background color name separated by a semicolon. For example, if the value of COLORFGBG is lightgray;blue, the default color object will be initialized to represent a lightgray foreground upon a blue background.

I added an experimental compile-time option --enable-colorfgbg to ncurses in July 2000. However it appears to be rarely used. The Cygwin, MinGW and SuSE packagers enable the feature; other packagers do not.

While I made improvements to the feature none was prompted by a user report. In occasional comments such as a mutt-dev thread Mutt 1.3.28 + ncurses 5.2 + xterm = blank screen I pointed out that a user's problem had already been fixed.

By the way, rereading the thread, I am reminded that slang's documentation for the feature is incomplete because it does not account for the xpm-configuration:

that's a bug that I fixed in September.  The problem was that when I coded
the $COLORFGBG logic (which btw is under-documented in rxvt - you have to
read the C code to see it), it didn't occur to me that its format might
change.  It happens that the format depends on whether xpm is linked in -
2 or 3 fields.  The background color is the last field.

(Since it's under-documented, it's also possible that in the future
anything that relies upon that format will be broken ;-).

One of the limitations with COLORFGBG is that it cannot compensate for color contrast problems (such as yellow-on-black becoming yellow-on-white). To compensate for that, the application would have to decide what choices of colors to use. That was why I added assume_default_colors. COLORFGBG can only solve part of the problem of optimizing screen-updates. It will not help the application (unless the application in turn checks the variable, decodes it properly and uses the information to make a set of colors with good contrast).

Unlike ncurses, slang may also have problems handling a mixture of explicit and default colors as indicated in a thread on slang-users in 2003, [slang-users] Strange COLORFGBG feature, where Davis commented

Nicolas Schodet <schodet+slang@xxxxxxxx> wrote:
>Why do get_default_colors return -1 if it do not find COLORFGBG or
>DEFAULT_COLOR ? Why not return 0 and fill fg_buf and bg_buf with a
>"default" default value ?

As far as I know, not all color terminals support the default color
escape sequence (ESC[39m), and there is no guarantee that sending a
terminal an escape sequence that it does not support will not have
an undesirable side effect.

Unfortunately, the terminfo database does not specify an escape
sequence to set _just_ the background to its default color.  Instead,
the terminfo database specifies the "op" string which sets both the
foreground and background color to the default:

COLORTERM

According to its (undated) change log, the $COLORTERM variable was introduced in 0.99-14. The patch files for slang 0.99-32 to 0.99-38 mention the feature in June 1996. Based on the DECUS tar files, 0.99-14 was likely mid-1995.

This is nonstandard. It is mentioned in the documentation, but no mention is made of values:

Of course not all terminals are color terminals. If the S-Lang global variable SLtt_Use_Ansi_Colors is non-zero, the terminal is assumed to be a color terminal. The SLtt_get_terminfo will try to determine whether or not the terminal supports colors and set this variable accordingly. It does this by looking for the capability in the terminfo/termcap database. Unfortunately many Unix databases lack this information and so the SLtt_get_terminfo routine will check whether or not the environment variable COLORTERM exists. If it exists, the terminal will be assumed to support ANSI colors and SLtt_Use_Ansi_Colors will be set to one. Nevertheless, the application should provide some other mechanism to set this variable, e.g., via a command line parameter.

COLORTERM_BCE

Added in 1.3.9, this is nonstandard and undocumented. It is mentioned only in changes for 1.3.9:

3. src/sl-feat.h: SLTT_XTERM_ALWAYS_BCE variable added to force the
   assumption of the bce (background-color-erase) capability of xterm.
   The default is 0, which means to accept the terminfo setting.
   To force it to 1 during run-time, set the COLORTERM_BCE environment
   variable.  This is useful when using, e.g., rxvt to login to a
   solaris system where the terminfo file will probably not indicate
   bce.

In other words, this tells slang to assume that the terminal actually supports bce. In earlier versions of slang this was a hard-coded assumption.

COLUMNS

Added in 0.99-21, this is standard (see ncurses manual page). Initially slang used this value, if set, after getting the number of columns from the terminal database (assumed to be termcap). Later, in 0.99-32, Davis stopped using termcap to give screen-size, and modified the initialization to start with results from TIOCGWINSZ ioctl-calls before and after reading the terminfo data.

DEFAULT_COLORS

Added in 0.99-36, this is nonstandard and undocumented. It is checked if COLORFGBG is not set, and performs the same function.

HOME

Added in 1.2.2, this is a standard variable (see ncurses manual page), but slang uses it to work with terminfo databases in $HOME/.terminfo (an ncurses extension)

LANG LC_ALL LC_CTYPE

Added in pre2-r0, this is standard.

LINES

Added in 0.99-21, this is standard (see ncurses manual page). Initially slang used this value, if set, after getting the number of lines from the terminal database (assumed to be termcap). Later, in 0.99-32, Davis stopped using termcap to give screen-size, and modified the initialization to start with results from TIOCGWINSZ ioctl-calls before and after reading the terminfo data.

TERM

Added in 0.99-7, this is standard (see ncurses manual page).

In standard usage, it is the name of an entry in the terminal database. Applications which rely upon hard-coded (i.e., nonstandard) behavior start by checking for specific values of TERM.

TERMCAP

Added in 0.99-21, this is nonstandard (see ncurses manual page).

Unlike other termcap applications, slang ignores TERMCAP values which point to a filename (see Termcap issues).

TERMINFO

Added in 0.99-11, this is standard (see ncurses manual page). Initially slang used this to fill in the first slot in an array of possible places to look for the terminal database:

#define MAX_TI_DIRS 3
static char *Terminfo_Dirs [MAX_TI_DIRS] =
{
   NULL,
   "/usr/lib/terminfo",
   "/usr/share/lib/terminfo"
};

Later (in 0.99-21), Davis added a fourth entry to the table, probably to work with ncurses on SunOS or Solaris:

#define MAX_TI_DIRS 4
static char *Terminfo_Dirs [MAX_TI_DIRS] =
{
   NULL,
   "/usr/lib/terminfo",
   "/usr/share/lib/terminfo",
   "/usr/local/lib/terminfo"
};

Davis continued to add to the table, mainly to check for the ncurses database. But the basic scheme is unchanged, slang assumes a directory tree. Because of that, slang's terminfo reader will not work with hashed-databases (added to ncurses in 2006).

WCWIDTH_CJK_LEGACY

Added in 2.1.4, this is nonstandard and undocumented. It is used to tell slang that Unicode ambiguous-width characters are double-width.

While ncurses recognizes more environment variables than slang, those are all documented. A few are of special interest:

NCURSES_ASSUMED_COLORS

Added in 2002:

        + add NCURSES_ASSUMED_COLORS environment variable, to allow users to
          override compiled-in default black-on-white assumption used in
          assume_default_colors().

one might suppose it to be the same as COLORFGBG. But it is not. The existing COLORFGBG feature uses names for the colors which makes it simple to use, but has the drawback that it is limited to 16 colors (plus “default”). The analogous ncurses feature which refers to color numbers does not have that limitation.

NCURSES_GPM_TERMS

Added in 2008, this has no counterpart in slang.

While ncurses supports both gpm- and xterm-mouse controls, the source for slang shows that only X10 protocol is supported:

int SLtt_set_mouse_mode (int mode, int force)
{
   if (force == 0)
     {
        char *term;

        if (NULL == (term = (char *) getenv("TERM"))) return -1;
        if (strncmp ("xterm", term, 5))
          return -1;
     }

   Mouse_Mode = (mode != 0);

   if (mode)
     tt_write_string ("\033[?9h");
   else
     tt_write_string ("\033[?9l");

   return 0;
}

Because the X10 protocol is a subset of the X11 xterm mouse protocol, ncurses could recognize that, e.g., with a terminal description that initializes the mouse with a different escape sequence with the user-defined capability XM. However, X10 protocol is considered obsolete because it does not distinguish between the buttons on the mouse as done in X11. Both of these came before 1990 (1986 and 1987, respectively).

Some slang users have asked for wheel mouse support. That would be a straightforward extension of the X11 mouse protocol (see the Colas Nahaboo X mouse wheel scroll page from 2000), provided by ncurses since 2005. The slang library source does not mention anything relevant to this request.

A few slang applications do more with the mouse, but they do not use slang to help with this.

NCURSES_NO_PADDING

Added in 1998:

        + add configure option --enable-no-padding, to allow environment
          variable $NCURSES_NO_PADDING to eliminate non-mandatory padding,
          thereby making terminal emulators (e.g., for vt100) a little more
          efficient (request by Daniel Eisenbud <eisenbud@cs.swarthmore.edu>).

the background for this was of course slang's discarding all of the padding and timing information. See for instance Timing/padding, as well as the discussion in Termcap issues.

NCURSES_NO_UTF8_ACS

Added in 2005:

        + implement environment variable NCURSES_NO_UTF8_ACS to support
          miscellaneous terminal emulators which ignore alternate character
          set escape sequences when in UTF-8 mode.

and a followup user-defined capability U8 in 2011:

        + improve the NCURSES_NO_UTF8_ACS feature by adding a check for an
          extended terminfo capability U8 (prompted by mailing list
          discussion).

this addresses the problem of certain terminal emulators which ignore the VT100-style shift-in/shift-out used for alternate character sets. While the environment variable provided configurability, putting the information in the terminal database was a better solution.

In contrast, since pre2-r0, slang ignores the terminal description if the locale encoding is UTF-8. Otherwise, slang ostensibly uses the terminal description but actually ignores the terminal database if the TERM variable begins with “xterm” or “aixterm” or “vt” followed by a number not equal to 52:

   if (strcmp(t, "vt52") && (*t++ == 'v') && (*t++ == 't')
       && (ch = *t, (ch >= '1') && (ch <= '9'))) Vt100_Like = 1;

   is_xterm = ((0 == strncmp (term, "xterm", 5))
               || (0 == strncmp (term, "rxvt", 4))
               || (0 == strncmp (term, "Eterm", 5)));

   almost_vtxxx = (Vt100_Like
                   || Linux_Console
                   || is_xterm
                   || !strcmp (term, "screen"));

TERMINFO

While the TERMINFO variable is standard, there is no standard interpretation of what it means. X/Open Curses issue 7 mentions the variable in A.3 Selecting a Terminal:

If the environment variable TERMINFO is defined, any program using Curses checks for a local
terminal definition before checking in the standard place.  For example, on implementations
which use the traditional directory layout for the terminfo data, if TERM is set to att4424, then
the compiled terminal definition is found in by default the path:

    a/att4424

within an implementation-specific directory.

That is, it refers to a directory-tree as “traditional” (I would say “conventional”) and does not prescribe a standard implementation. When I implemented the hashed-database implementation, I kept the pathname semantics close to the directory-tree, but slang cannot read hashed databases.

TERMINFO_DIRS

Added in 1996.

While slang's configure script has (since 2007) used the --terminfo option of ncurses scripts ncurses5-config or ncursesw5-config to obtain a suitable default for TERMINFO, it still relies upon a hard-coded set of directories rather using the corresponding --terminfo-dirs option to specify additional directories that ncurses's packager may have found necessary.

Finally, with the release of ncurses 6 in August 2015, those script names have become stale. Use ncurses6-config or ncursesw6-config when available.

Keyboard issues

While both ncurses and slang read from the terminal database, users (e.g., of Lynx and other programs) would like some way to configure key bindings that does not involve updating the terminal database.

Lynx is unlike Mutt and Midnight Commander because it uses features of ncurses or slang to accomplish this goal.

Lynx has tables of special keys (such as cursor down), to which it assigns a code (using a range that cannot be mistaken for ordinary characters). Any of the special-characters or ordinary characters can be “bound” to Lynx's repertoire of functions using these internally assigned character-codes. The letter “k” for instance is bound to the KEYMAP function which shows something like this:

←←←                                                  Current Key Map (p1 of 6)→
a           ADD_BOOKMARK  add to your personal bookmark list                  ↑
A           ADDRLIST      like LIST command, but always shows the links' URLs ▮
b           PREV_PAGE     view the previous page of the document              ▮
c           CREATE        create a new file or directory                      ▮
C           CHDIR         change current directory                            ▮
d           DOWNLOAD      download the current link to your computer          ▒
e           EDIT          edit the current document or a form's textarea      ▒
E           ELGOTO        edit the current link's URL or ACTION and go to it  ▒
f           DIRED_MENU    display a full menu of file operations              ▒
g           GOTO          go to a document given as a URL                     ▒
G           ECGOTO        edit the current document's URL and go to it        ▒
h           HELP          display help on using the browser                   ▒
i           INDEX         display an index of potentially useful documents    ▒
j           JUMP          go directly to a target document or action          ▒
k           KEYMAP        display the current key map                         ▒
K           EDITMAP       display the current edit-key map                    ▒
l           LIST          list the references (links) in the current document ▒
m           MODIFY        modify the name or location of a file or directory  ▒
n           NEXT          search for the next occurrence                      ▒
N           PREV          search for the previous occurrence                  ▒
o           OPTIONS       display and change option settings                  ▒
p           PRINT         display choices for printing the current document   ↓
-- press space for more, use arrow keys to move, '?' for help, 'q' to quit.

Lynx also has functions for decoding the sequence of characters which make up a special key. In practice, those are useful for systems which do not have a curses (or slang) library and a terminal database. When Lynx is able to use a terminal database, those functions simply translate from curses (or slang) key-codes into Lynx's key symbols.

The users would like these keys to be more easily configurable. We discussed this more than once on the Lynx mailing list:

LYNX-DEV LYNX low level key mapping - Problems (February 1998)
LYNX-DEV Key re-mapping (April 1998).

In the first discussion, Davis indicated that a solution would be slang specific:

When I find time I hope to modify lynx to use the slang keymap code.
This would eliminate the huge switch statements and clean up the that
portion of the code quite a bit.  Unfortunately, it would be
slang-specific and would not work with curses.

Resuming the discussion in April, John Davis said that the keymap feature in Lynx should be rewritten (to take advantage of slang functions), and I agreed, suggesting that he go ahead and provide an initial implementation which I would further modify for curses.

slang

Davis' proposed changes two weeks later used slang functions for modifying the definitions assigned to key-symbols in slang.

Those functions first appeared in 0.99-36, apparently part of the “Improved curses emulation” by Michael Elkins.

The patch re-uses part of slang's initialization which is done before reading the terminal database. Here is the relevant code from slang:

   /* Now add most common ones. */
#ifndef IBMPC_SYSTEM
   SLkm_define_keysym ("^@", 0, Keymap_List);

   SLkm_define_keysym ("\033[A", SL_KEY_UP, Keymap_List);
   SLkm_define_keysym ("\033OA", SL_KEY_UP, Keymap_List);
   SLkm_define_keysym ("\033[B", SL_KEY_DOWN, Keymap_List);
   SLkm_define_keysym ("\033OB", SL_KEY_DOWN, Keymap_List);
   SLkm_define_keysym ("\033[C", SL_KEY_RIGHT, Keymap_List);
   SLkm_define_keysym ("\033OC", SL_KEY_RIGHT, Keymap_List);
   SLkm_define_keysym ("\033[D", SL_KEY_LEFT, Keymap_List);
   SLkm_define_keysym ("\033OD", SL_KEY_LEFT, Keymap_List);
   SLkm_define_keysym ("\033[F", SL_KEY_END, Keymap_List);
   SLkm_define_keysym ("\033OF", SL_KEY_END, Keymap_List);
   SLkm_define_keysym ("\033[H", SL_KEY_HOME, Keymap_List);
   SLkm_define_keysym ("\033OH", SL_KEY_HOME, Keymap_List);
   SLkm_define_keysym ("\033[2~", SL_KEY_IC, Keymap_List);
   SLkm_define_keysym ("\033[3~", SL_KEY_DELETE, Keymap_List);
   SLkm_define_keysym ("\033[5~", SL_KEY_PPAGE, Keymap_List);
   SLkm_define_keysym ("\033[6~", SL_KEY_NPAGE, Keymap_List);
   SLkm_define_keysym ("\033[7~", SL_KEY_HOME, Keymap_List);
   SLkm_define_keysym ("\033[8~", SL_KEY_END, Keymap_List);
#else

That assigns one or more definitions to each of the key symbols for the editing- and cursor-keypads on a PC keyboard. The two definitions correspond to normal and application keys. The extra definition is redundant in a terminal description because curses applications initialize the terminal when they start up. Davis uses both in slang expecting the terminal to be uninitialized.

Later, slang retrieves information from the terminal database, using the idiom "^(xx)" to denote the termcap capability xx:

#ifdef REAL_UNIX_SYSTEM
   strcpy (esc_seq, "^(kX)");
   for (i = 0; i <= 9; i++)
     {
        esc_seq[3] = '0' + i;
        SLkm_define_keysym (esc_seq, SL_KEY_F(i), Keymap_List);
     }
   SLkm_define_keysym ("^(k;)", SL_KEY_F(10), Keymap_List);
   SLkm_define_keysym ("^(F1)", SL_KEY_F(11), Keymap_List);
   SLkm_define_keysym ("^(F2)", SL_KEY_F(12), Keymap_List);

   SLkm_define_keysym ("^(ku)", SL_KEY_UP, Keymap_List);
   SLkm_define_keysym ("^(kd)", SL_KEY_DOWN, Keymap_List);
   SLkm_define_keysym ("^(kl)", SL_KEY_LEFT, Keymap_List);
   SLkm_define_keysym ("^(kr)", SL_KEY_RIGHT, Keymap_List);
   SLkm_define_keysym ("^(kP)", SL_KEY_PPAGE, Keymap_List);
   SLkm_define_keysym ("^(kN)", SL_KEY_NPAGE, Keymap_List);
   SLkm_define_keysym ("^(kh)", SL_KEY_HOME, Keymap_List);
   SLkm_define_keysym ("^(@7)", SL_KEY_END, Keymap_List);
   SLkm_define_keysym ("^(K1)", SL_KEY_A1, Keymap_List);
   SLkm_define_keysym ("^(K3)", SL_KEY_A3, Keymap_List);
   SLkm_define_keysym ("^(K2)", SL_KEY_B2, Keymap_List);
   SLkm_define_keysym ("^(K4)", SL_KEY_C1, Keymap_List);
   SLkm_define_keysym ("^(K5)", SL_KEY_C3, Keymap_List);
   SLkm_define_keysym ("^(%0)", SL_KEY_REDO, Keymap_List);
   SLkm_define_keysym ("^(&8)", SL_KEY_UNDO, Keymap_List);
   SLkm_define_keysym ("^(kb)", SL_KEY_BACKSPACE, Keymap_List);
   SLkm_define_keysym ("^(@8)", SL_KEY_ENTER, Keymap_List);
   SLkm_define_keysym ("^(kD)", SL_KEY_DELETE, Keymap_List);
#endif

Both tables have been modified since 0.99-36:

The first table now predefines normal/application mode settings for xterm's Home and End keys; the original dealt only with rxvt.
The second table now retrieves the definitions of function keys 11 and 12; the original retrieved only 10 numbered function keys.

ncurses

The changes made for Lynx to use SLkm_define_keysym do not have an exact equivalent in X/Open Curses. However, I can extend ncurses to do useful features which cannot be done easily by an application using the library.

Lynx was not the only application that could use such a feature, nor was this the first time that the need had been established. I already had the necessary piece in ncurses from the previous year:

970531
        ...
        + modify mousemask() to use keyok() to enable/disable KEY_MOUSE, so
          that applications can disable ncurses' mouse and supply their own
          handler.
        + add extensions keyok() and define_key().  These are designed to allow
          the user's application better control over the use of function keys,
          e.g., disabling the ncurses KEY_MOUSE.  (The define_key idea was from
          a mailing-list thread started by Kenneth Albanowski
          <kjahds@kjahds.com> Nov'1995).

and updating Lynx followed:

1999-02-16 (2.8.2dev.17)
...
* function expand_substring used with NCURSES for user-defined mapping keysyms
  to byte sequences was parsing the "^(...)" construct wrong - KW
1998-09-12 (2.8.1dev.26)
...
* use ncurses define_key to implement lynx-keymaps (see dev.10) - TD
* modify slang keymap code to work with slang 0.99-38 - TD
...
1998-08-15 (2.8.1dev.21)
...
* The current keymapping for the DOS Slang ports maps the keys directly to lynx
  actions via keymap.c.  There are several places in the code that test for
  arrow keys, page up or down, home and end for special handling that depend on
  the lynx definition of the keys.  Hence, the keys in the DOS Slang ports were
  failing the tests and not working as expected.  The following patch now maps
  these special keys.  They can no longer be mapped individually via the KEYMAP
  mechanism in lynx.cfg.  Arrows should now work in GOTO fields.  Tested the
  patch against the binaries maade by makefile.dos, makefile.dsl, and
  makefile.wsl - DK
* make keymaps file consistently named on Unix and VMS - TD
1998-06-04 (2.8.1dev.14)
* change 'keymap[] array to unsigned char, move logic that relies on its size
  into LYKeymap.c - TD
1998-05-10 (2.8.1dev.10)
...
* added keymaps/keysym patch (it really should be integrated with the normal
  lynx.cfg file (John E Davis).

Timing/Padding

slang

In 1994, the first version of slang to provide screen management removed padding and time-delay information from the terminal description. When necessary, Davis invented his own equivalents as discussed in screen management.

Additionally, slang provided default values for terminal capabilities based on the name of the terminal. For example “vt100” meant only a terminal emulator (apparently Davis no longer used anything else). Thus:

Depending on the environment, the terminal can support ANSI color.
Color support assumed bce (background color erase).
The terminal does not need padding or time delays.
The terminal has a PC-style editing keypad with Insert, Delete, Home, End, PageUp and PageDown.

A terminfo description for vt100 looks like this:

vt100|vt100-am|dec vt100 (w/advanced video),
        am, mc5i, msgr, xenl, xon,
        cols#80, it#8, lines#24, vt#3,
        acsc=``aaffggjjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~,
        bel=^G, blink=\E[5m$<2>, bold=\E[1m$<2>,
        clear=\E[H\E[J$<50>, cr=\r, csr=\E[%i%p1%d;%p2%dr,
        cub=\E[%p1%dD, cub1=^H, cud=\E[%p1%dB, cud1=\n,
        cuf=\E[%p1%dC, cuf1=\E[C$<2>,
        cup=\E[%i%p1%d;%p2%dH$<5>, cuu=\E[%p1%dA,
        cuu1=\E[A$<2>, ed=\E[J$<50>, el=\E[K$<3>, el1=\E[1K$<3>,
        enacs=\E(B\E)0, home=\E[H, ht=^I, hts=\EH, ind=\n, ka1=\EOq,
        ka3=\EOs, kb2=\EOr, kbs=^H, kc1=\EOp, kc3=\EOn, kcub1=\EOD,
        kcud1=\EOB, kcuf1=\EOC, kcuu1=\EOA, kent=\EOM, kf0=\EOy,
        kf1=\EOP, kf10=\EOx, kf2=\EOQ, kf3=\EOR, kf4=\EOS, kf5=\EOt,
        kf6=\EOu, kf7=\EOv, kf8=\EOl, kf9=\EOw, lf1=pf1, lf2=pf2,
        lf3=pf3, lf4=pf4, mc0=\E[0i, mc4=\E[4i, mc5=\E[5i, rc=\E8,
        rev=\E[7m$<2>, ri=\EM$<5>, rmacs=^O, rmam=\E[?7l,
        rmkx=\E[?1l\E>, rmso=\E[m$<2>, rmul=\E[m$<2>,
        rs2=\E<\E>\E[?3;4;5l\E[?7;8h\E[r, sc=\E7,
        sgr=\E[0%?%p1%p6%|%t;1%;%?%p2%t;4%;%?%p1%p3%|%t;7%;%?%p4%t;5
            %;m%?%p9%t\016%e\017%;$<2>,
        sgr0=\E[m\017$<2>, smacs=^N, smam=\E[?7h, smkx=\E[?1h\E=,
        smso=\E[7m$<2>, smul=\E[4m$<2>, tbc=\E[3g,

Since slang is a termcap application, it ignores sgr. Also, it ignores all of the termcap boolean and numeric capabilities other than colors. Finally, it has its own notion of how to handle special keys. Here are the capabilities that slang's initialization sees, counting its built-in modifications:

acsc, blink, bold, civis, clear, cnorm, colors, csr, cub, cub1, cud, cud1, cuf, cuf1, cup, cuu, cuu1, dch1, dl, dsl, ed, el, el1, enacs, flash, fsl, il, ka1, ka3, kb2, kbs, kc1, kc3, kcub1, kcud1, kcuf1, kcuu1, kdch1, kend, kent, kf0, kf1, kf10, kf11, kf12, kf2, kf3, kf4, kf5, kf6, kf7, kf8, kf9, khome, knp, kpp, krdo, kund, op, rev, ri, rmacs, rmcup, rmir, rmkx, rmso, setab, setaf, setb, setf, sgr0, smacs, smcup, smir, smkx, smso, smul, tsl,

and the resulting terminal description:

vt100|vt100-am|dec vt100 (w/advanced video),
        acsc=``aaffggjjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~,
        bel=^G, blink=\E[5m, bold=\E[1m, clear=\E[H\E[J, cr=\r,
        csr=\E[%i%p1%d;%p2%dr, cub=\E[%p1%dD, cub1=^H, cud=\E[%p1%dB,
        cud1=\n, cuf=\E[%p1%dC, cuf1=\E[C, cup=\E[%i%p1%d;%p2%dH,
        cuu=\E[%p1%dA, cuu1=\E[A, dch1=\E[P, dl=\E[%dM, ed=\E[J,
        el=\E[K, el1=\E[1K, enacs=\E(B\E)0, il=\E[%dL, ka1=\EOq,
        ka3=\EOs, kb2=\EOr, kbs=^H, kc1=\EOp, kc3=\EOn, kcub1=\EOD,
        kcud1=\EOB, kcuf1=\EOC, kcuu1=\EOA, kend=\E[8~, kent=\EOM,
        kf0=\EOy, kf1=\EOP, kf10=\EOx, kf2=\EOQ, kf3=\EOR, kf4=\EOS,
        kf5=\EOt, kf6=\EOu, kf7=\EOv, kf8=\EOl, kf9=\EOw,
        khome=\E[7~, kich1=\E[2~, knp=\E[6~, kpp=\E[5~, rev=\E[7m,
        ri=\EM, rmacs=^O, rmir=\E[4l, rmkx=\E[?1l\E>, rmso=\E[m,
        sgr0=\E[m\017, sitm=\E[3m, smacs=^N, smam=\E[?7h, smir=\E[4h,
        smkx=\E[?1h\E=, smso=\E[7m, smul=\E[4m, kcub1=\EOD,
        kcud1=\EOB, kcuf1=\EOC, kcuu1=\EOA, kdch1=\E[3~,
        kend=\EOF, khome=\EOH,

The omission of sitm from slang's capabilities is not an error (in this commentary). It corresponds to a hard-coded assignment in slang.

If the COLORTERM environment happens to be set, the “vt100” supports ANSI color. Even without that, according to infocmp, many of the capabilities are different.

The slang “vt100” has changed over the years. Comparing with 0.99-38"

the Home and End keys differed (see discussion)
there is some support for non-bce terminals (see discussion).

But the treatment of time-delays in the terminal description is unchanged.

ncurses

Although pcurses (the ancestor of ncurses supported padding, there were improvements to make. I did not get involved with that, initially. Among others, Andrey A. Chernov made improvements, e.g,. to handle fractional values in the delay times.

I first started developing padding code when discussing fixes for minicom in 1996 with Miquel van Smoorenburg, as well as working on vttest (which uses padding to work with the DEC hardware terminals).

Late in 1997, I started making fixes to this area, prompted by lots of discussion with Daniel Weaver. Here is an example:

From danw@znyx.com  Tue Nov 11 22:05:43 1997
Received: from mail1.geo.net (mail1.geo.net [166.90.101.11])
        by mail.clark.net (8.8.8/8.8.8) with SMTP id WAA18824
        for <dickey@clark.net>; Tue, 11 Nov 1997 22:05:38 -0500 (EST)
Received: (qmail 24293 invoked from network); 12 Nov 1997 03:04:25 -0000
Received: from electra.znyx.com (root@209.0.10.2)
  by mail1.geo.net with SMTP; 12 Nov 1997 03:04:25 -0000
Received: from kimba (danw.znyx.com [209.0.10.36]) by electra.znyx.com (8.7.6/8.7.3) with SMTP id TAA07874 for <dickey@clark.net>; Tue, 11 Nov 1997 19:01:28 -0800
Message-Id: <3.0.3.32.19971111190123.009bd320@209.0.10.2>
X-Sender: danw@209.0.10.2
X-Mailer: QUALCOMM Windows Eudora Pro Version 3.0.3 (32)
Date: Tue, 11 Nov 1997 19:01:23 -0800
To: "T.E.Dickey" <dickey@clark.net>
From: Daniel Weaver <danw@znyx.com>
Subject: Re: ncurses-4.1-971108.patch.gz
In-Reply-To: <199711120046.TAA04422@clark.net>
References: <3.0.3.32.19971111161136.00916990@209.0.10.2>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Status: RO
Content-Length: 1951
Lines: 51

At 07:46 PM 11/11/97 -0500, T.E.Dickey wrote:
>> At 09:23 PM 11/8/97 -0500, T.E.Dickey wrote:
>> >ncurses 4.1 - patch 971108 - T.Dickey <dickey@clark.net>
>> >
>> The trailpad code does look correct and I will be testing that later
>> this week.
>thanks - I figure you'll be looking closer at it than other people.

You got that right.  My ftp-release of TACK still fails to gather
feedback.  I know its being downloaded.  Maybe people would rather
be pack rats instead of guinea pigs.

>
>ok - I did this stuff last, and it probably needs closer review.

Did that.  My ultimate goal is to make ncurses work with an RS-323
terminal without XON.  It will be glorious day.

>(I looked briefly at initc, but ran out of time - will continue this
>weekend).

No problem.  This can wait a few days.

>
>The XSI standard is vague - goes into no more detail on delays than
>the Solaris manpages (which also are vague).  I read through that, but
>didn't find any reason to not change the code this way (but by comparison
>testing, maybe something will show up).

Padding is so misunderstood.  Just for a laugh, look at the vt220
terminfo.  The ind cap has 20 ms of mandatory pad.  This was done
be a desperate man.  I see it over and over again.  People put the
padding in the wrong place.  TACK will put an end to that.

Once we get tputs() fixed we will have to go through the terminfo
file and fix some of these errors.  I don't own a vt220 so I will
have to rely on someone else to do the work.

>It got rid of a cast (and the code is 100 bytes smaller).  I don't much
>like casts, because it suppresses error checking.

Agreed.

On another front, I am in contact with RMS.  He is trying to convince me,
at my encouragement, to put TACK under GPL.  He claims we can do that
without forcing ncurses under GPL.  So far I see no compelling reason to
use GPL.  I will keep you updated.

-----------------------------
Dan Weaver, ZNYX Corporation
danw@znyx.com

Although slang's removal of padding information was not new, it took a while before it became an issue. A few users stated that scrolling in mutt was slower with ncurses than with slang. Eventually those reports went away. However

Early on (until 1996), ncurses scrolling was known to be slow.
Eric Raymond adapted Heckel's (comparison) algorithm to provide an improvement called “hashmap”
I benchmarked hashmap, found that it did not actually improve performance, and disabled the feature.
Many digressions later, Alexander V. Lukyanov provided several improvements to hashmap in 1997 that essentially made it work as well as Raymond claimed the original version did.
After releasing ncurses 4.2, I changed the configure-script in March 1998 so that hashmap was no longer considered “experimental” and (after more delays) most users got that improvement with ncurses 5.0 in 1999.

Complicating things, I had a bug report in the middle of this long process where the apparent problem was that the terminal description used time-delays for the indexing (ind and ri) controls used for scrolling.

This discussion with Daniel Eisenbud prompted me to implement the NCURSES_NO_PADDING environment variable:

From eisenbud@cs.swarthmore.edu Sun May  3 21:16:16 1998
Received: from allspice.cs.swarthmore.edu (allspice.cs.swarthmore.edu [130.58.68.10])
        by ice.clark.net (8.8.8/8.8.8) with ESMTP id VAA04867
        for <dickey@clark.net>; Sun, 3 May 1998 21:16:16 -0400 (EDT)
Received: by allspice.cs.swarthmore.edu (VMailer, from userid 653)
        id 4A21B3EBF; Sun,  3 May 1998 21:15:37 -0400 (EDT)
Message-ID: <19980503211537.A702@cs.swarthmore.edu>
Date: Sun, 3 May 1998 21:15:37 -0400
From: Daniel Eisenbud <eisenbud@cs.swarthmore.edu>
To: "T.E.Dickey" <dickey@clark.net>
Subject: Re: Glitch in hashmap with ncurses 4.2?
References: <19980503194501.A27643@cs.swarthmore.edu> <199805040106.VAA08736@shell.clark.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Mailer: Mutt 0.92.5i
In-Reply-To: <199805040106.VAA08736@shell.clark.net>; from T.E.Dickey on Sun, May 03, 1998 at 09:06:22PM -0400

Status: OR

On Sun, May 03, 1998 at 09:06:22PM -0400, T.E.Dickey <dickey@clark.net> wrote:
> > set.  With ncurses 4.2 and vt100, the whole lines that the arrow is
> > moving on and off of (one or both - I'm not sure which, though I can
> > check) get completely redrawn, instead of just the first three
> > characters of each line, for the arrow, getting redrawn.  From what you
> > said above, and what I could easily see of the typescripts, you were
> > scrolling the whole screen with "<" and ">" instead, right?  Or am I
> > now confused?
> ok - I can see it, but do not (yet) know the cause.  I saw that it does
> this with vt100, but not xterm, so I made changes to a copy of vt100
> description until the problem went away.  The changes that did this are the
> delay times on the ed, el, el1, sgr0, rev capabilities (or some subset of
> them - it'll take me more time to isolate).  Perhaps there is something
> amiss with the cost-computation that tells ncurses which type of update
> strategy to use (or maybe it's right - but not obvious:  hardware terminals
> can go slowly for some operations, so ncurses compensates by chosing one of
> a half-dozen strategies for each of 3-4 areas).  The delay times (in
> milliseconds) are the numbers in brackets.  (In any case, I'll study this
> some more to make sure I understand why it is repainting the whole line)
>
> -- slang doesn't implement the delays (perhaps 1.x does).  so it would
> not treat vt100 and xterm differently.

Cool - thanks for looking into this!

On NCSA telnet on slow macs, it is definitely more costly to reprint the
whole line - it makes a noticeable flicker.  Similarly, over a slow
modem connection, this makes scrolling a whole lot slower.  Are the cost
computations based on the behavior of an actual hardware vt100?  I bet a
very tiny portion of the people whose term type is vt100 are actually
sitting at a real vt100 terminal these days.

If it is based on the hardware, is there some easy way to determine
accurate timings to optimize for a particular terminal emulator?  Also,
it would be nice to have a way to tell ncurses apps that one is on a low
bandwidth connection, so that optimization for smallest amount of data
is much more important than optimization for supposed hardware terminal
speed...  Maybe an environment variable?

-Daniel

--
Daniel Eisenbud
eisenbud@cs.swarthmore.edu

At the time, I did not realize that Eisenbud's configuration was a special case. There are several types of time-delays in a terminfo description:

mandatory delays
non-mandatory delays, used when flow-control (XON/XOFF) is supported
non-mandatory delays, used when flow-control (XON/XOFF) is not supported

The “vt100” terminal description uses flow-control. However, the terminal description for NCSA Telnet does not. The ncsa description (from ncurses) used by Eisenbud has a non-mandatory delay for the forward-indexing operation, but no flow-control:

ind=\n$<150*>,

Pausing for 150 milliseconds after each line limits the forward scrolling to less than 7 lines per second, requiring 3.6 seconds to scroll 24 lines.

It is possible that NCSA Telnet did not actually require the time delay (comments in a posting to gnu.emacs.help XON/XOFF flow control (was Re: Re-mapping Keys) indicate that it responded to the corresponding control S/Q), but it is unlikely that anyone still uses the program and can verify if changing that feature would work well. Development on NCSA Telnet stopped January 1, 1995.

In the context of Mutt, that also provided motivation to look closely at slang and notice (or be reminded) that slang discards all time-delays.

After that, I was well aware that slang did not implement padding (see my comment in the slang vs. ncurses thread on comp.unix.programmer in July 1998).

One of the reasons for changing the status of hashmap after releasing ncurses 4.2 was that we considered it still a development version. This email to Alexander Lukyanov touches on several relevant points:

From dickey Mon Feb  1 15:01:29 1999
Subject: Re: cursor at exit
To: lav@yars.free.net (Alexander V. Lukyanov" <lav@yars.free.net>)
Date: Mon, 1 Feb 1999 15:01:29 -0500 (EST)
In-Reply-To: <19990201213159.A2886@gemini.yars.free.net>  from "Alexander V. Lukyanov" <lav@yars.free.net>" at Feb 1, 99 09:31:59 pm
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text
Status: RO
Content-Length: 2168
Lines: 50

>
> On Mon, Feb 01, 1999 at 12:53:03PM -0500, dickey@clark.net wrote:
> > > That depends on what to call "use". Ncurses could use them if a program
> > > called wscrl on screen-wide window. That happens quite rarely in practice.
> >
> > I was using wscrl since late 1995 because scrolling didn't work well - 
> > which I told Eric about repeatedly - and finally gave up trying to get him 
> > to fix it.  Other people (mc for example) simply stopped using ncurses. 
>
> BTW, I beleave they stopped using it because slang feels faster on linux
> console. What if we could make a driver for mmapped linux console :)

No - they stated that they stopped using it because Zeyd did not respond to
their requests for resizable windows (there was some type of sigwinch
implementation before 1.8.6 but it dumped core, and was removed).  (I had
them remove some of the wording when I gave them a patch for ncurses 4.1 +
patches, because it was written in the present tense as though 1.8.6 was
current, but some of it's still in the source code).

But yes - I understand that different types of terminals (from serial) is
what you are proposing.

However, much of the "faster" is simply configuration (there are a lot of
people who set $TERM to "vt100" and do not understand why it's slower than
the correct "xterm" - but since slang ignores all padding and delays it is
"faster" though it would not work for a real vt100).

> > I don't remember the details, but do remember fixing some logic related to
> > the sc/rc code there between 1.9.9e/1.9.9g, e.g., in 1996 (I can check if
> > it's of importance).
>
> No, it's not too important (to me anyway - I do remember something like
> that).
no problem (I tend to remember changes I made).

anyway - if no new bugs (like the ^Z) are reported this week, I'll go back
to the documentation.  I've only about one weekend's work to complete a
pre-release.  I have old/known bugs, but unless I find a fix, will wait
for after packaging a release before continuing (if they were simple, I
would have already fixed them).

>
>    Alexander.

--
Thomas E. Dickey
dickey@clark.net
http://www.clark.net/pub/dickey

However as shown by my comments, I had not seen that “vt100” (for most users) did not necessarily use the time delays except as part of the strategy for updating the screen. This reply (likely also talking about NCSA Telnet) says the same thing:

From dickey Fri Feb 12 22:14:28 1999
Subject: Re: ncurses scrolling
To: vikasa@writeme.com (Vikas Agnihotri <vikasa@writeme.com>)
Date: Fri, 12 Feb 1999 22:14:28 -0500 (EST)
In-Reply-To: <19990212211929.A13821@ems.att.com>  from "Vikas Agnihotri <vikasa@writeme.com>" at Feb 12, 99 09:19:29 pm
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text
Status: RO
Content-Length: 1113
Lines: 30

>
> This is regarding Mutt, using ncurses 4.2, scrolling poorly when I log
> in over a 28.8 modem from home. With the exact same setup, Mutt+SLang
> scrolls quite smoothly. Everything else the same, when I switch to
> Mutt+ncurses, the scrolling is atrocious. I can literally see the
> entire screen being re-painted. Of course, when I am at work, on my
> local LAN, the difference is not as pronounced, but, mind you, S-lang
> still scrolls *much* more smoothly than ncurses.
>
> Why so? Does slang use some specific shortcuts/cheats that ncurses
> doesnt?
two things: did you turn on hashmap?  and are you using $TERM=vt100?

hashmap is optional with 4.2, but we've turned it on as a default
in the post-4.2 patches.

vt100 is an example of a terminal description that has substantial time
delays in the standard description.  that's to accomodate real vt100's
(not emulators ;-).  I added an environment variable $NCURSES_NO_PADDING
last summer to turn off these delays.  (slang ignores all delays).

>
> Thanks,
> Vikas

--
Thomas E. Dickey
dickey@clark.net
http://www.clark.net/pub/dickey

Thus, as long as only terminal emulators are considered, slang's removal of time-delays is mostly harmless. For a hardware terminal directly connected to a serial port, the results would not be good.

Time delays would be used for hardware terminals lacking flow-control, or connected through a modem which does not provide that feature. Sometimes, as in a thread on comp.os.linux.development.apps entitled gotoxy(x,y) in Linux? people may confuse the two:

On 18 Dec 1999 11:33:27 +0300, Victor Wagner <victor@wagner.rinet.ru>
wrote:

>Even slang, second popular console toolkit is not universal. Its authors
>think "Hey, nobody use hardware terminals now. Why would we bother with
>XON/XOFF flow control". And they effectively cut lot of people from
>their user base.

I am not sure what you are referring to.  S-Lang's SLang_init_tty
functions allows the control of XON/XOFF flow control.  In fact slrn,
which uses slang has a `use_flow_control' variable that the user may
set to control this aspect.

--John

For what it's worth, I helped correct the mistake:

> I am not sure what you are referring to.  S-Lang's SLang_init_tty
> functions allows the control of XON/XOFF flow control.  In fact slrn,
> which uses slang has a `use_flow_control' variable that the user may
> set to control this aspect.

I assumed he was confused about terminology:  XON/XOFF versus interfaces
that require padding (like the machines we have down the hall at work with
a serial terminal plugged in to eliminate a graphics monitor ;-)

--
Thomas E. Dickey
dic...@clark.net
http://www.clark.net/pub/dickey

SLcurses by the way turns off the flow-control feature that Davis referred to. That is another difference between SLcurses and curses.

Termcap issues

Although slang is usually run on a system which provides it with a terminfo-database, it uses termcap capability names to refer to the terminal data.

The irony of that is illustrated by this bug report which made some minor changes to slang to allow it to provide a termcap-function interface:

Debian Bug report logs - #51176 slang: simple additions for termcap emulation

That change glosses over the differences between termcap- and terminfo-interfaces:

while termcap and terminfo data are syntactically similar, the latter can have arithmetic and logical expressions which make the meaning radically different, and
real termcap applications may rely upon tgetent loading the content of the terminal description into a buffer provided by the application.

For the former, both provide the tputs and tgoto functions which know about the differences in syntax. In slang, Davis started by assuming termcap, then added some special cases to make it work with terminfo. Padding and timing are some of those special cases.

The latter is used rarely now, but used to be common with termcap applications. The feature is used in xterm to provide a TERMCAP environment variable which sets the size (lines and columns capabilities) for applications which do not handle SIGWINCH. It is useful only for systems where xterm happens to be linked with a real termcap library. Otherwise, xterm can only provide the size information. The other data are omitted.

XTerm

In xterm, the decision whether to update TERMCAP is made at compile time. In the early 1990s, some systems used BSD semantics, while others used SysV semantics. The former were assumed to set TERMCAP. The imake definitions for each system chose one or the other. Early work on Linux used BSD sources, e.g., for X applications. While working on ncurses, it was common to have both ncurses and termcap libraries for development. Some packagers (Davis did not specify which system) chose to replace the termcap library with a link to ncurses.

In this case, xterm got an empty buffer for the termcap data, and added the lines and columns values, resulting in something like this:

TERMCAP=:li#24:co#80:

Notwithstanding his comment, that is a valid termcap. Most users would not find it very useful, of course.

Bear in mind that Davis frequently complained about problems with terminal descriptions, without providing details on what the actual problem was, and “solved” the problem by hard-coding behavior into slang. Davis alluded to this problem in email, but provided no useful information beyond what you can read here:

From davis@space.mit.edu Fri Sep  6 15:45 EDT 1996
Received: from space.mit.edu (SPACE.MIT.EDU [18.75.0.10]) by mail.Clark.Net (8.7.3/8.6.5) with SMTP id PAA01708 for <dickey@clark.net>; Fri, 6 Sep 1996 15:45:55 -0400 (EDT)
Received: from wiwaxia.mit.edu by space.mit.edu AA24686; Fri, 6 Sep 96 15:44:11 EDT
Received: by wiwaxia AA28816; Fri, 6 Sep 96 15:44:10 EDT
Message-Id: <199609061945.PAA01708@mail.Clark.Net>
Date: Fri, 6 Sep 96 15:44:10 EDT
From: "John E. Davis" <davis@space.mit.edu>
To: dickey@clark.net
Subject: Re: xterm problems
X-Mailer: Jed [0.98-0]
Content-Type: text
Content-Length: 342
Status: RO

>all, the existing termcap package works fairly well, and it didn't occur
>to me that someone would be sucked in by the noise that Eric was making.

The reason that I brought it up is that several people have reported
to me that my programs will not work with that xterm because of the,
IMO, its invalid TERMCAP environment variable.

--John

He made the same comments in other places, still without the useful information which would have related the problem to the termcap emulation of ncurses:

Re: Xterm and TERMCAP mentioning 0.99-35 (1996/10/09).
LYNX-DEV Re: Slang version of lynx doesn't work in xterm in XFree86 3.2 (1996/11/10).

That is, he never showed the value of the environment variable. Another user did mention this on the debian-user mailing list late in October 1996 (Re: xterm-debain termcap?), but unless you subscribed to the (high-volume) list, you would never see the comment:

Zlatko Rek wrote:

I have problems with xterm. When it is opened the TERM and TERMCAP
environment variables are set to:

  TERM=xterm
  TERMCAP=co#80:li#24:

I use Jed editor and if I want to edit file the Terminal not powerful
enough for SLang. message appears. When TERMCAP is unset Jed works OK.

At the time, by the way, I did not use Debian. I developed using Slackware (which still provides a real termcap library).

You can read Davis' solution in the changes for 0.99-35 (October 1996):

3. If terminal is xterm, the TERMCAP environment variable will be
   ignored.  This change was necessary because of the braindead
   TERMCAP setting created by the latest version xterm.

The actual change added the 1 ifdef'd chunk in tcap_tgetent. Incidentally, the accompanying comment about “tc=” contradicts the logic in the fragment shown here, and is irrelevant anyway:

   if (SLtt_Try_Termcap == 0) return -1;
#if 1
   /* XFREE86 xterm sets the TERMCAP environment variable to an invalid
    * value.  Specifically, it lacks the tc= string.
    */
   if (!strncmp (term, "xterm", 5))
     return -1;
#endif
   termcap = (unsigned char *) getenv ("TERMCAP");
   if ((termcap == NULL) || (*termcap == '/')) return -1;

   /* SUN Solaris 7&8 have bug in tset program under tcsh,
    * eval `tset -s -A -Q` sets value of TERMCAP to ":",
    *  under other shells it works fine.
    *  SUN was informed, they marked it as duplicate of bug 4086585
    *  but didn't care to fix it... <mikkopa@cs.tut.fi>
    */
   if ((termcap[0] == ':') && (termcap[1] == 0))
     return -1;

   /* We have a termcap so lets use it provided it does not have a reference
    * to another terminal via tc=.  In that case, use terminfo.  The alternative
    * would be to parse the termcap file which I do not want to do right now.
    * Besides, this is a terminfo based system and if the termcap were parsed
    * terminfo would almost never get a chance to run.  In addition, the tc=
    * thing should not occur if tset is used to set the termcap entry.
    */
   ti->flags = SLTERMCAP;

   t = termcap;
   while ((len = tcap_extract_field (t)) != -1)
     {
        if ((len > 3) && (t[0] == 't') && (t[1] == 'c') && (t[2] == '='))
          return -1;
        t += (len + 1);
     }

If Davis had shown the value of $TERMCAP (and given the comments, perhaps he never looked), the problem should be obvious to anyone who has developed termcap applications. Starting in August 1996, on the XFree86 mailing list, we discussed the problem of providing a binary for xterm which would avoid this problem. David Dawes' release plan for XFree86 3.2 included these points:

It's time to get XFree86 3.2 finalised.  We'd like to see the code
complete by mid October, and ready to ship by the end of October.  Our
main priority now is to fix problems in the 3.1.2G version, add features
required for planned enhancements of XF86Setup, and aim for stability.
Completely new code should be limited to self-contained areas which
could easily be disabled if problems arise.  We don't want any major
changes to well-established areas.  The deadline for new code will be
the end of September.
...
Essential items:

1. Documentation.

2. Resolve xdm shadow vs non-shadow passwords for Linux/Elf.

3. Resolve xterm ncurses vs termcap for Linux/Elf -- I think the current
   plan is to link against libtermcap statically.  This need to be tested.

Later, in xterm patch #38, I provided a fix in my initial configure script for xterm:

dnl ---------------------------------------------------------------------------
dnl Check for tgetent function in termcap library.  If we cannot find this,
dnl we'll use the $LINES and $COLUMNS environment variables to pass screen
dnl size information to subprocesses.  (We cannot use terminfo's compatibility
dnl function, since it cannot provide the termcap-format data).
AC_DEFUN([CF_FUNC_TGETENT],
[
AC_MSG_CHECKING(for workable tgetent function)
AC_CACHE_VAL(cf_cv_func_tgetent,[
cf_save_LIBS="$LIBS"
cf_cv_func_tgetent=no
for cf_termlib in termcap termlib curses
do
        LIBS="$cf_save_LIBS -l$cf_termlib"
        AC_TRY_RUN([
/* terminfo implementations ignore the buffer argument, making it useless for
* the xterm application, which uses this information to make a new $TERMCAP
*/
int main()
{
        char buffer[1024];
        buffer[0] = 0;
        tgetent(buffer, "vt100");
        exit(buffer[0] == 0); }],
        [cf_cv_func_tgetent=$cf_termlib
         break],
        [AC_TRY_LINK([],[tgetent(0, 0)],[break])],
        [echo trying link
         AC_TRY_LINK([],[tgetent(0, 0)],
                [cf_cv_func_tgetent=$cf_termlib
                 break],
                [echo link failed
                 LIBS="$cf_save_LIBS"])])
done
])
AC_MSG_RESULT($cf_cv_func_tgetent)
])dnl

Line-drawing

While it is rare to encounter an application which misbehaves because of the missing termcap description assumed via a call to tgetent, an application using terminfo to emulate termcap can run into other problems.

Unlike terminfo, which has set_attributes (sgr), termcap has no comparable feature. However, when the developers of terminfo and curses during the 1980s assigned an equivalent termcap feature for exit_attribute_mode (sgr0), they chose a termcap capability me which has a slightly different meaning:

sgr sets (or resets) several attributes including the alternate character set.
sgr0 resets all of the attributes which may be set using sgr
termcap applications assume that the alternate character set is not set using me.

The screen program is a termcap application which makes that assumption. Michael Schroeder (one of its maintainers) pointed out this (and some other) termcap emulation issues:

20010908
        + modify tgetent() to check if exit_attribute_mode resets the alternate
          character set, and if so, attempt to adjust the copy of the termcap
          "me" string which it will return to eliminate that part.  In
          particular, 'screen' would lose track of line-drawing characters
          (report by Frederic L W Meunier <0@pervalidus.net>, analysis by
          Michael Schroeder).

Besides adjusting sgr0, ncurses also adjusts cancelled capabilities so that for instance tgetstr will return a null pointer. slang also does this although the actual return values depend on a flag set as a side-effect via SLtt_initialize which says whether to use termcap or terminfo conventions. None of that is is documented. Here is slang's documentation for SLtt_initialize in 2.3.1a:

  12.34.  int SLtt_initialize (char *);

      Synopsis
        ??

      Usage
        int SLtt_initialize (char *);

      Description
        ??

      See Also
        ??

You must read slang's source code to use this interface. Likewise, you must keep in mind the specific version of slang, otherwise you will run into problems with compatibility, as hinted by this item from the changes for 2.2.3:

103. src/sldisply.c: Added some termcap compatibility functions that
     were ifdefed out in v2.2.2 but were made available by some linux
     distributions for code to use: SLtt_tgetent, SLtt_tgoto,
     SLtt_tputs (patch forwarded by Marco Atzeri).

Besides the workarounds in tgetent, I have also found it necessary to modify the terminal database:

20040327
        ...
        + add a check in tic for terminfo entries having an sgr0 but no sgr
          string.  This confuses Tru64 and HPUX curses when combined with
          color, e.g., making them leave line-drawing characters in odd places.

By adding a check in tic, it is possible to find these special cases and devise terminal descriptions which will work for different implementations of curses.

Although slang is a termcap application, it does not make similar fixes to the terminal description.

Terminfo format

Portable(sic)

Rather than use the terminfo library functions which are standard, Davis chose to write his own terminfo reader. In a posting to comp.terminals he mentioned that, along with a misconception:

Newsgroups: comp.terminals,comp.sys.hp.hpux
Path: cs.utk.edu!cssun.mathcs.emory.edu!emory!gatech!bloom-beacon.mit.edu
      !senator-bedfellow.mit.edu!usenet
From: davis@space.mit.edu
Subject: Re: termcap entry
Date: 8 May 1995 13:27:35 GMT
Organization: Center for Space Research
Message-ID: <3ol687$2rq@senator-bedfellow.MIT.EDU>
References: <1995Apr7.152134.25551@ka4ybr.com> <3mu0l7$obe@tadpole.fc.hp.com>
        <3mv3rv$mj4@vixen.cso.uiuc.edu> <3oarad$3p3@senator-bedfellow.MIT.EDU>
        <3ol2m5$643@hpuamsa.neth.hp.com>
Reply-To: davis@space.mit.edu
NNTP-Posting-Host: wiwaxia.mit.edu

In article <3ol2m5$643@hpuamsa.neth.hp.com>,
franks@neth.hp.com (Frank Slootweg) writes:
 : davis@space.mit.edu wrote:
 : > Actually, terminfo compiled files are supposed to be portable across any
 : > system.  So, `untic-ing' it is unnecessary--- just copy the fole to the
 : > other system.
 :
 :   Are you sure? As far as I know, compiled terminfo files contain
 : multi-byte binary data. I doubt that that binary data is bi-endian (i.e.
 : big versus little endian) safe, but could of course be wrong.

I am pretty sure about it.  I wrote my own terminfo reader for my JED editor
and it runs fine on all kinds of Unix systems.  Nowhere in the code do I
check to see what byte ordering of the machine uses.

I wrote it straight from the manpage.  In fact, I have included all relevant
parts of the man page as comments in the source file.  If you want to look
at the source, pick up slang.tar.gz from space.mit.edu:/pub/slang.  Then
look at the file slang/src/sltermin.c.

--John

The term manual page describes the format of a compiled terminfo description but does not actually say that the result is “supposed to be portable across any system” (there is a difference between format and content).

The ncurses term(5) manual page was updated to add this comment (seen in 1.9.7, October 1995):

Despite the consistent use of little-endian for numbers and the otherwise
self-describing format, it is not wise to count on portability of binary
terminfo entries between commercial UNIX versions.  The problem is that there
are at least three versions of terminfo (under HP-UX, AIX, and OSF/1) which
diverged from System V terminfo after SVr1, and have added extension
capabilities to the string table that (in the binary format) collide with
System V and XSI Curses extensions.  See \fBterminfo\fR(\*n) for detailed
discussion of terminfo source compatibility issues.

My mail from 1995 mentions more than one person who might have made that observation, but no mention of it was made in the ncurses NEWS file. I recall discovering (by a core dump) that AIX 3.2.5 and AIX 4.3 did not work with the same binary terminfo files, but the report which prompted Eric or Zeyd to make that change to the manual could have been made by Larry Schwimmer or Tim Mooney.

I had that in mind a few years later when I commented on a bug report for slang on AIX:

From dickey Fri Sep 17 09:42:58 1999
From: "T.E.Dickey" <dickey@shell.clark.net>
Subject: Re: Salng, ncurses and .terminfo
Newsgroups: alt.lang.s-lang
References: <7rpnqm$hn6$1@nnrp1.deja.com> <039E3.2756$zI3.50482@iad-read.news.verio.net> <slrn7u38hc.kf2.davis@aluche.mit.edu>
Organization: Clark Internet Services, Inc., Ellicott City, MD USA
User-Agent: tin/pre-1.4-19990805 ("Sumerland") (UNIX) (SunOS/5.6 (sun4u))
Status: RO
Content-Length: 993
Lines: 23

John E. Davis <davis@space.mit.edu> wrote:
> On Thu, 16 Sep 1999 16:39:24 GMT, T.E.Dickey <dickey@shell.clark.net>
> wrote:
>>This is the first time I recall someone attributing problems in slang
>>to a (mis)feature of ncurses.

> If an incorrect terminfo file gets installed and used, then that will
> affect a slang-based application.  In particular, if the terminfo file
> contains an incorrect size for, e.g., the string capability table,
> then a malloc error could occur.  I cannot explain this problem unless
> something like this is happening.

It did not occur to me immediately - but perhaps you are not aware that
AIX 3.2.5's terminfo format is(was) not compatible with the code in slang
that reads terminfo files.  AIX 4.x is (I understand) compatible with the
SVr4 format.

-- Likewise Digital Unix, iirc (an example of a system that's X/Open
   certified but incompatible in an area that X/Open oversees ;-).

--
Thomas E. Dickey
dickey@clark.net
http://www.clark.net/pub/dickey

although at that time I no longer had access to the AIX systems. Later, someone offered me access to a set of Unix machines which I used to investigate this issue. That did not include AIX 3.2.5, but did include several variations where the offsets in the compiled terminfo tables were different. I added that information to ncurses by making customized versions of the Caps file which Eric Raymond had added in April 1995:

20020706
        + add Caps.hpux11, as an example.
...
20020112
        + add Caps.osf1r5, as an example.
...
20010804
        + add Caps.keys example for experimental extended function keys
          (adapted from a patch by Ilya Zakharevich).
        ...
        + ignore blank lines in Caps* files in MKkey_defs.sh script (report by
          Albert Chin-A-Young).
        + correct definition of key_end in Caps.aix4, which left KEY_END
          undefined (report by Albert Chin-A-Young).
...
20010616
        + add parentheses in macros for malloc in test.priv.h, fixes an
          expression in view.c (report by Wolfgang Gutjahr <gutw@knapp.co.at>).
        + add Caps.uwin, as an example.
        + change the way curses.h is generated, making the list of function
          key definitions extracted from the Caps file.
...
20010526
        + add experimental --with-caps=XXX option to customize to similar
          terminfo database formats such as AIX 4.x
        + add Caps.aix4 as an example.
        + modify Caps to add columns for the the KEY_XXX symbols.

That is, it appeared in ncurses 5.3 (October 2002).

Adapting from ncurses

Davis was aware of my work, as indicated by his response to a bug report several years later, and adapted it for slang in 2.2.3 (December 2010). His comment in changes.txt says this:

Changes since 2.2.2
1. src/sltermin.c: Added support for native terminfo binary layouts.
   Some systems have binary terminfo files that are incompatible with
   the standard ncurses layout.  Use the --terminfo=layout configure
   option to specify a non-standard layout (hpux11, osf1r5, uwin,
   aix4, default).  This incompatibity was pointed out by River Tarnell.

but src/terminfo/README gives more relevant information:

The include files in this directory define the binary layout of the terminfo
file.  They were derived from the Cap files distributed with
ncurses-5.6 using the parsecaps.sl script.

Those files contain the following copyright notice:
...

Reading the comment about “standard” you may see that the confusion between terminfo format and content was still present. The format was defined long ago because it came from one implementation, while the content evolved over several years with different development paths. X/Open Curses does not define the format; it lists capabilities (without specifying their organization) which make up the content.

When Eric Raymond added Caps, he was using a Dell SVr4 PC (not widely known, see Dell UNIX Lives Again! page for instance), Zeyd apparently used IBM AIX, while I was developing on several platforms including other SVr4 systems (mainly Sun Solaris and SGI IRIX64, but also HP-UX, AIX and OSF/1). There were some differences between his early versions of Caps and Solaris; I reported those and Eric made changes to use the same ordering, same names. There were still some differences which one may see in the current Caps file:

ncurses defines several capabilities named with “OT” prefixes. Those came from Eric's so-called “master” format for representing termcap features which had no counterpart in X/Open Curses.
Solaris uses an older full name for one of the printer capabilities that is documented in X/Open Curses (micro_char_size versus micro_col_size).
Neither Solaris or X/Open Curses documented termcap names for several other printer capabilities. Eric invented names for these, e.g., “Xh” for “hhlm” (enter_horizontal_hl_mode).

A diff between the tables for ncurses and Solaris shows 42 lines changed. That difference is for the closest match between ncurses and other implementations. The ncurses compiled terminfo happens to work with Solaris because Solaris does not care about the extra data, and because the format (the way data is read) is the same. For other systems, the format is the same (due to the common origin for some, imitation for others), but the order of capabilities differs. There is no standard which gives that order.

As an example, consider OSF/1 (later Tru64), like ncurses, a separate implementation written based on documentation. Ignoring the order shows that OSF/1 recognized a few dozen capabilities (mostly special keys) not documented in X/Open Curses:

caps.csv | 566 +++++++======================================================
1 file changed, 68 insertions(+), 1 modification(!), 497 unchanged lines(=)

But taking into account the order of capabilities, the difference is much larger:

caps.csv | 585 +++++++++--!!!!!!!!!!!!!!!!!==================================
1 file changed, 87 insertions(+), 19 deletions(-), 161 modifications(!), 318 unchanged lines(=)

One difference which should affect slang is that the position of bce is different. But slang largely ignores it, due to prevailing hard-coded logic that checks the value of the TERM environment variable. This slice of source-code illustrates that point:

#if SLTT_HAS_NON_BCE_SUPPORT
   Can_Background_Color_Erase = TGETFLAG ("ut");   /* bce */
   /* Modern xterms have the BCE capability as well as the linux console */
   if (Can_Background_Color_Erase == 0)
     {
        Can_Background_Color_Erase = (Linux_Console
# if SLTT_XTERM_ALWAYS_BCE
                                      || is_xterm
# endif
                                      );
     }
#endif

User-defined

On the other hand, Davis has not adapted the ncurses user-defined capabilities feature which I added early in 1999. Several possibilities come to mind:

It is not “standard” (i.e., not contemplated in the SGI term or terminfo manual pages in 1994).
The slang library already has something to solve the problem.

Most of the extended capabilities use names longer than two characters and consequently are unavailable to termcap applications. This part of Davis' 2009 response regarding the Caps files is relevant:

I will look into providing an option to use a specified binary layout.
It is possible to allow the format to be specified at run-time instead
of assuming a statically specified layout, but I am not sure that will
actually be useful.

When I implemented user-defined capabilities in ncurses in 1999, I did this to deal with a nuisance: people citing termcap as better than terminfo because the latter was predefined, and could not include user-defined capabilities. In practice few people used the termcap extensibility, but I found a good use for the feature in terminfo: special (i.e., function-, cursor-, keypad-) key definitions.

Originally, xterm had a termcap describing 12 function keys. But users wanted to be able to use modifiers (shift-, control-, alt-) to produce different keyboard sequences. Wanting something and being able to produce a useful design are different matters:

One could simply add to the table of predefined strings. The various Unix (and application) developers did that. It only became more cumbersome.
One could make the modifiers part of the curses interface. PDCurses took this route (and does not use terminfo or termcap), by defining symbols for the more useful combinations of key and modifier. Doing that interferes with portability to other curses implementations.
In ncurses I chose to provide the ability to define capabilities using just the name (and relying on syntax to determine the type). Application developers could choose meaningful names.

Over time, I used a few of these extended capabilities for ncurses (see the user_caps(5) manual page), but most of the capabilities are intended for applications.

Termcap has two limitations which are relevant:

Conventional termcap descriptions are limited to 1023 bytes, and
termcap capability names are two characters.

While some termcap implementations dispense with the former limit, the latter is not going away. That allows about nine thousand capability names, which you might assume would allow enough useful mnemonics. In practice, that is not enough characters. Here is the predefined list of termcap special keys:

&9 &0 *1 *2 *3 *4 *5 *7 *8 *9 *0 #1 #2 #3 #4

%b %a %c %d %f %e %g %j %i %h !1 !2 !3 K1 K3

K2 @1 kb K4 K5 @2 kB @3 kC @4 @5 @6 kt kl kd

kr ku kD kL kS kE @7 @8 @9 k0 k1 k2 k3 k4 k5

k6 k7 k8 k9 k; F1 F2 F3 F4 F5 F6 F7 F8 F9 FA

FB FC FD FE FF FG FH FI FJ FK FL FM FN FO FP

FQ FR FS FT FU FV FW FX FY FZ Fa Fb Fc Fd Fe

Ff Fg Fh Fi Fj Fk Fl Fm Fn Fo Fp Fq Fr @0 %1

kh kT kI kA kF kH Km %4 %2 %3 kN %5 %6 %7 kP

%9 %8 %0 &1 &5 &2 kR kM &3 &4 &6 *6 &7 ka &8

In contrast, the terminfo names are longer, but easier to understand. Here is a corresponding list, in the same order:

kBEG kCAN kCMD kCPY kCRT kDC kDL kEND kEOL kEXT kFND kHLP kHOM kIC kLFT

kMOV kMSG kNXT kOPT kPRT kPRV kRDO kRES kRIT kRPL kSAV kSPD kUND ka1 ka3

kb2 kbeg kbs kc1 kc3 kcan kcbt kclo kclr kcmd kcpy kcrt kctab kcub1 kcud1

kcuf1 kcuu1 kdch1 kdl1 ked kel kend kent kext kf0 kf1 kf2 kf3 kf4 kf5

kf6 kf7 kf8 kf9 kf10 kf11 kf12 kf13 kf14 kf15 kf16 kf17 kf18 kf19 kf20

kf21 kf22 kf23 kf24 kf25 kf26 kf27 kf28 kf29 kf30 kf31 kf32 kf33 kf34 kf35

kf36 kf37 kf38 kf39 kf40 kf41 kf42 kf43 kf44 kf45 kf46 kf47 kf48 kf49 kf50

kf51 kf52 kf53 kf54 kf55 kf56 kf57 kf58 kf59 kf60 kf61 kf62 kf63 kfnd khlp

khome khts kich1 kil1 kind kll kmous kmov kmrk kmsg knp knxt kopn kopt kpp

kprt kprv krdo kref kres krfr kri krmir krpl krst ksav kslt kspd ktbc kund

Taking that into account, I chose to add function-key definitions for xterm's modified keys using names derived from the standard terminfo names. Again, the user_caps(5) manual page explains the details.

Currently, a quarter of xterm's terminal description consists of user-defined capabilities (the different-colored string capabilities beginning with Cr). None of that is used by slang:

#       Reconstructed via infocmp from file: /usr/local/ncurses/share/terminfo/x/xterm
xterm|xterm terminal emulator (X Window System),
        OTbs, am, bce, km, mc5i, mir, msgr, npc, xenl, AX, XT,
        colors#8, cols#80, it#8, lines#24, pairs#64,
        acsc=``aaffggiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~,
        bel=^G, blink=\E[5m, bold=\E[1m, cbt=\E[Z, civis=\E[?25l,
        clear=\E[H\E[2J, cnorm=\E[?12l\E[?25h, cr=\r,
        csr=\E[%i%p1%d;%p2%dr, cub=\E[%p1%dD, cub1=^H,
        cud=\E[%p1%dB, cud1=\n, cuf=\E[%p1%dC, cuf1=\E[C,
        cup=\E[%i%p1%d;%p2%dH, cuu=\E[%p1%dA, cuu1=\E[A,
        cvvis=\E[?12;25h, dch=\E[%p1%dP, dch1=\E[P, dim=\E[2m,
        dl=\E[%p1%dM, dl1=\E[M, ech=\E[%p1%dX, ed=\E[J, el=\E[K,
        el1=\E[1K, flash=\E[?5h$<100/>\E[?5l, home=\E[H,
        hpa=\E[%i%p1%dG, ht=^I, hts=\EH, ich=\E[%p1%d@,
        il=\E[%p1%dL, il1=\E[L, ind=\n, indn=\E[%p1%dS,
        invis=\E[8m, is2=\E[!p\E[?3;4l\E[4l\E>, kDC=\E[3;2~,
        kEND=\E[1;2F, kHOM=\E[1;2H, kIC=\E[2;2~, kLFT=\E[1;2D,
        kNXT=\E[6;2~, kPRV=\E[5;2~, kRIT=\E[1;2C, kb2=\EOE, kbs=^?,
        kcbt=\E[Z, kcub1=\EOD, kcud1=\EOB, kcuf1=\EOC, kcuu1=\EOA,
        kdch1=\E[3~, kend=\EOF, kent=\EOM, kf1=\EOP, kf10=\E[21~,
        kf11=\E[23~, kf12=\E[24~, kf13=\E[1;2P, kf14=\E[1;2Q,
        kf15=\E[1;2R, kf16=\E[1;2S, kf17=\E[15;2~, kf18=\E[17;2~,
        kf19=\E[18;2~, kf2=\EOQ, kf20=\E[19;2~, kf21=\E[20;2~,
        kf22=\E[21;2~, kf23=\E[23;2~, kf24=\E[24;2~,
        kf25=\E[1;5P, kf26=\E[1;5Q, kf27=\E[1;5R, kf28=\E[1;5S,
        kf29=\E[15;5~, kf3=\EOR, kf30=\E[17;5~, kf31=\E[18;5~,
        kf32=\E[19;5~, kf33=\E[20;5~, kf34=\E[21;5~,
        kf35=\E[23;5~, kf36=\E[24;5~, kf37=\E[1;6P, kf38=\E[1;6Q,
        kf39=\E[1;6R, kf4=\EOS, kf40=\E[1;6S, kf41=\E[15;6~,
        kf42=\E[17;6~, kf43=\E[18;6~, kf44=\E[19;6~,
        kf45=\E[20;6~, kf46=\E[21;6~, kf47=\E[23;6~,
        kf48=\E[24;6~, kf49=\E[1;3P, kf5=\E[15~, kf50=\E[1;3Q,
        kf51=\E[1;3R, kf52=\E[1;3S, kf53=\E[15;3~, kf54=\E[17;3~,
        kf55=\E[18;3~, kf56=\E[19;3~, kf57=\E[20;3~,
        kf58=\E[21;3~, kf59=\E[23;3~, kf6=\E[17~, kf60=\E[24;3~,
        kf61=\E[1;4P, kf62=\E[1;4Q, kf63=\E[1;4R, kf7=\E[18~,
        kf8=\E[19~, kf9=\E[20~, khome=\EOH, kich1=\E[2~,
        kind=\E[1;2B, kmous=\E[M, knp=\E[6~, kpp=\E[5~,
        kri=\E[1;2A, mc0=\E[i, mc4=\E[4i, mc5=\E[5i, meml=\El,
        memu=\Em, op=\E[39;49m, rc=\E8, rep=%p1%c\E[%p2%{1}%-%db,
        rev=\E[7m, ri=\EM, rin=\E[%p1%dT, ritm=\E[23m, rmacs=\E(B,
        rmam=\E[?7l, rmcup=\E[?1049l, rmir=\E[4l, rmkx=\E[?1l\E>,
        rmm=\E[?1034l, rmso=\E[27m, rmul=\E[24m, rs1=\Ec,
        rs2=\E[!p\E[?3;4l\E[4l\E>, sc=\E7, setab=\E[4%p1%dm,
        setaf=\E[3%p1%dm,
        setb=\E[4%?%p1%{1}%=%t4%e%p1%{3}%=%t6%e%p1%{4}%=%t1%e%p1%{6}%=%t3%e%p1%d%;m,
        setf=\E[3%?%p1%{1}%=%t4%e%p1%{3}%=%t6%e%p1%{4}%=%t1%e%p1%{6}%=%t3%e%p1%d%;m,
        sgr=%?%p9%t\E(0%e\E(B%;\E[0%?%p6%t;1%;%?%p5%t;2%;%?%p2%t;4%;%?%p1%p3%|%t;7%;%?%p4%t;5%;%?%p7%t;8%;m,
        sgr0=\E(B\E[m, sitm=\E[3m, smacs=\E(0, smam=\E[?7h,
        smcup=\E[?1049h, smir=\E[4h, smkx=\E[?1h\E=,
        smm=\E[?1034h, smso=\E[7m, smul=\E[4m, tbc=\E[3g,
        u6=\E[%i%d;%dR, u7=\E[6n, u8=\E[?%[;0123456789]c,
        u9=\E[c, vpa=\E[%i%p1%dd, Cr=\E]112\007,
        Cs=\E]12;%p1%s\007, E3=\E[3J, Ms=\E]52;%p1%s;%p2%s\007,
        Se=\E[2 q, Ss=\E[%p1%d q, kDC3=\E[3;3~, kDC4=\E[3;4~,
        kDC5=\E[3;5~, kDC6=\E[3;6~, kDC7=\E[3;7~, kDN=\E[1;2B,
        kDN3=\E[1;3B, kDN4=\E[1;4B, kDN5=\E[1;5B, kDN6=\E[1;6B,
        kDN7=\E[1;7B, kEND3=\E[1;3F, kEND4=\E[1;4F,
        kEND5=\E[1;5F, kEND6=\E[1;6F, kEND7=\E[1;7F,
        kHOM3=\E[1;3H, kHOM4=\E[1;4H, kHOM5=\E[1;5H,
        kHOM6=\E[1;6H, kHOM7=\E[1;7H, kIC3=\E[2;3~, kIC4=\E[2;4~,
        kIC5=\E[2;5~, kIC6=\E[2;6~, kIC7=\E[2;7~, kLFT3=\E[1;3D,
        kLFT4=\E[1;4D, kLFT5=\E[1;5D, kLFT6=\E[1;6D,
        kLFT7=\E[1;7D, kNXT3=\E[6;3~, kNXT4=\E[6;4~,
        kNXT5=\E[6;5~, kNXT6=\E[6;6~, kNXT7=\E[6;7~,
        kPRV3=\E[5;3~, kPRV4=\E[5;4~, kPRV5=\E[5;5~,
        kPRV6=\E[5;6~, kPRV7=\E[5;7~, kRIT3=\E[1;3C,
        kRIT4=\E[1;4C, kRIT5=\E[1;5C, kRIT6=\E[1;6C,
        kRIT7=\E[1;7C, kUP=\E[1;2A, kUP3=\E[1;3A, kUP4=\E[1;4A,
        kUP5=\E[1;5A, kUP6=\E[1;6A, kUP7=\E[1;7A, rmxx=\E[29m,
        smxx=\E[9m,

Extended numbers

While working on tctest in 2011, I noticed a termcap entry with a number too large to translate into terminfo. That was added to FreeBSD in 1995. Technically the entry itself was too large as well (1094 bytes) because it was formatted with one capability per line. Formatted conventionally it is 838 bytes:

pcvtXX|pcvt vt200 emulator (DEC VT220):\
        :AL=\E[%dL:DC=\E[%dP:DL=\E[%dM:DO=\E[%dB:IC=\E[%d@:LE=\E[%dD:\
        :RI=\E[%dC:SF=\E[%dS:SR=\E[%dT:UP=\E[%dA:\
        :ac=llmmkkjjuuttvvwwqqxxnnpprr``aa:\
        :ae=^O:al=\E[L:am:as=^N:bl=^G:bs:cb=\E[1K:cd=\E[J:ce=\E[K:\
        :cl=\E[H\E[J:cm=\E[%i%d;%dH:cr=^M:cs=\E[%i%d;%dr:ct=\E[3g:\
        :dc=\E[P:dl=\E[M:do=^J:eA=\E)0:ei=\E[4l:ho=\E[H:im=\E[4h:\
        :it#8:k1=\E[17~:k2=\E[18~:k3=\E[19~:k4=\E[20~:k5=\E[21~:\
        :k6=\E[23~:k7=\E[24~:k8=\E[25~:kD=\E[3~:kH=\E[4~:kI=\E[2~:\
        :kN=\E[6~:kP=\E[5~:kb=\177:kd=\EOB:ke=\E[?1l\E>:kh=\E[1~:\
        :kl=\EOD:km:kr=\EOC:ks=\E[?1h\E=:ku=\EOA:le=^H:mb=\E[5m:\
        :md=\E[1m:me=\E[m:mi:mr=\E[7m:ms:nd=\E[C:pb#16000000:pt:\
        :rc=\E8:rf=/usr/share/tabset/vt100:\
        :rs=\Ec\E>\E[?3l\E[?4l\E[?5l\E[?7h\E[?8h:\
        :sc=\E7:se=\E[27m:sf=\ED:so=\E[7m:sr=\EM:st=\EH:ue=\E[24m:\
        :up=\E[A:us=\E[4m:vt#3:xn:

The pb#16000000 is a problem for conventional terminfo, because the number is bigger than a signed 16-bit integer (32767). Referring to the description of pb

lowest baud rate where padding needed

it is not a realistic value because:

There is no system providing a 16-million baud rate. When the terminal description was added in 1995; currently the largest definitions might be a few million on Linux. For FreeBSD, the maximum is less: 921600 (still larger than 32767).
The terminal description does not use padding or time delays, making the pb capability a waste of space.

However (like user-definable capabilities), it was something to consider if for no other reason than to demonstrate an extension for the tiny niche of end-users who would demand it.

An extension should not break existing applications using ncurses:

Applications which use ncurses's termcap and terminfo function interfaces would be unaffected by an extension for bigger numbers, since their “integer” return value on a 32-bit machine is large enough.
Applications using the terminfo data structure seen by including <term.h> would only be able to use the legacy 16-bit numbers (since binary compatibility must be preserved). That is not a drawback, because there are no realistic termcap applications which could use larger numbers.

The hybrid termcap/terminfo “solution” for emacs to use 24-bit color mentioned in the FAQ is no longer a termcap application but rather an application which has introduced a problem with portability.

Davis chose not to use the termcap or terminfo function interfaces (although they are standardized). Rather, slang uses the conventional (not standardized) binary format. Because it is impossible to extend the conventional binary format in a way which makes extended numbers readable by a legacy application, any format which provides extended numbers would not be readable by slang. In that case, the only consideration necessary for ncurses is to design the extension so that it will not be a nuisance to slang users, i.e., do not dump core.

Unicode

I recall a discussion on the Lynx mailing list saying that because I am the Lynx maintainer I am obligated to make improvements to the slang configuration because some users prefer slang.

Not so. Free software is self-service; if you need a feature, you might get it faster if you do the work.

I stated my policy several times on the mailing list, as in Re: lynx-dev LYNX: should nested <blockquotes> indent even further?

On Sat, Nov 30, 2002 at 03:52:21PM +0900, Henry Nelson wrote:
> > It doesn't work with slang (would require a lot of work, since slang doesn't
> > manage that sort of thing).
>
> I certainly agree with the "require a lot of work."
>
> Still, "most" (same author as slang, for those who don't know) has one
> of the best horizontal scroll implementations I know of for a text viewer.
> It's intuitive and configurable (and works with multibyte characters).

as a matter of policy I don't use "most".  John Davis orphaned it for most
of a year some time ago, did not accept patches to fix a couple of core
dumps.  He (or other people interested in slang) are welcome to submit
patches to lynx, but I don't foresee any circumstance where I'll do more
on my part than repair damage that I may have done to lynx's slang code.

For instance, when I rewrote the popup-window feature (used with EXTERN_PAGE and the options menu), I made certain that it worked with the slang configuration. That is an example of repairing damage.

On the other hand, modifying the slang configuration to handle UTF-8 in Lynx is not repair, but continuing development. Because there have been no updates from Davis since shortly after slang was modified to support UTF-8, it is natural to presume that just calling SLutf8_enable will not magically make Lynx's slang configuration give good results.

Lynx stores webpages internally as a UTF-8 string, but when formatting the output it must compute the proper places in that string to wrap at the right margin, as well as determining the width of Unicode values that may print as two cells on the screen. When it fails to do this, the results are poor.

Here are screenshots with ncurses and slang configurations showing a page from Markus Kuhn's UTF-8 demo:

The screenshots show several places where the lines are wrapped too early in the slang configuration (on the right).

Improving the right-margin logic was only part of the development activity which I have done with Lynx. The options menu appearance relies upon padding with blanks and uses the ncurses library to help with that process. Lacking that help, the results in the slang configuration are unsurprisingly poor.

Again, here are screenshots to illustrate, using ru_RU.UTF-8 for the locale:

Lynx also has to handle multibyte characters on input. Again that takes development work, which like all development, gets bug reports (see for example Debian Bug report logs - #841155 Lynx browser: Typing utf-8 2-byte characters only diplays first). There have been few recent bug reports for the slang configuration because few packagers use that.

Compatibility

Early on, slang acquired a reputation for changing incompatibly with each release. My experience with most was not an isolated occurrence. Around the same time, Michael Allen Dorman commented on the Debian developer's mailing list in the S-lang availabe from my ftp site for testing thread:

OK, I remember this now.  But S-Lang is exceptional in that the C
interface changes every release or two, i.e., every couple of months.
Hence each new version produces incompatible libraries with previous
versions.

The 1.2.2 release came with instructions for upgrading, running to 295 lines (more than a thousand words). Reading through it, no mention is made of the screen management functions. Users of S-Lang were not so lucky.

The 2.0.0 release also came with instructions for upgrading, provided as appendices to the language (167 lines, 790 words) and C runtime documents (179 lines, 630 words). In the appendix, Davis noted that he changed the data type of SLsmg_Char_Type in <slang.h>used for storing information about a cell on the screen, and modified a half-dozen function prototypes passing this information to use the new data type. Thus, upgrading to 2.0.0 required modifying some programs that use the screen management functions.

However, he also changed other types (such as SLtype), increasing their size (and meaning). Upgraded programs might compile, but then require debugging. For instance after upgrading Lynx, I found a bug in slang (reported as a followup to Gisle Vanem's build-fix), and cited in slang's changes.txt:

Changes since 2.0.0 pre-release 4
...
3. src/slsmg.c: Bug fix affecting SLsmg_Newline_Behavior (Thomas
   Dickey).

Development timeline

S-Lang's changelog has no dates (only version numbers). Its Git repository only goes back to May 2006. For earlier records, there are only tar files.

I have a collection of tar files, because I have had to build slang to test with Lynx and other programs:

You can find the more recent ones on Davis' website or ftp site.
Davis does not provide any of the 1.3.x tar files, nor 1.4.x until 1.4.9;
I found some on an OpenBSD site,
and others in source-RPMs made by Hao Li at CalTech.
For the older ones, the DECUS repository is helpful.

Here is a list of version numbers mentioned in the changelog and the date, where known (either I have a tar file, or email mentioning it):

2.3.1a (2016/11/24), tar file
2.3.1 (2016/10/30), tar file
2.3.0 (2014/09/17), tar file
2.2.4 (2011/04/10), tar file
2.2.3 (2010/12/13), tar file
2.2.2 (2009/12/05), tar file
2.2.1 (2009/09/06), tar file
2.2.0 (2009/08/02), tar file
2.1.4 (2008/09/07), tar file
2.1.3 (2007/10/28), tar file
2.1.2 (2007/09/09), tar file
2.1.1 (2007/06/30), tar file
2.1.0 (2007/05/28), tar file
2.0.7 (2006/11/04), tar file
2.0.6 (2006/02/06), tar file
2.0.5 (2005/10/17), tar file
2.0.4 (2005/06/20), tar file
2.0.3 (2005/05/23), tar file
2.0.2 (2005/05/16), tar file
2.0.0 (2005/04/25), tar file
pre2-r4 (2005/02/07), tar file
pre2-r3 (2004/12/14), tar file
pre2-r2 (2004/11/27), tar file
pre2-r0 (2004/10/11), tar file
1.4.9 (2003/03/23), tar file
1.4.8 (2003/01/26), diff file
1.4.7 (2002/12/08), diff file
1.4.6 (2002/10/07), diff file
1.4.5 (2002/02/10), diff file, tar file from OpenBSD
1.4.4 (2001/02/20), diff file, tar file from OpenBSD
1.4.3 (2001/02/04), diff file
1.4.2 (2000/08/19), diff file, tar file from OpenBSD
1.4.1 (2000/04/22), diff file, tar file from OpenBSD
1.4.0 (2000/01/09), diff file, tar file from OpenBSD
1.3.10 (1999/11/14), tar file
1.3.9 (1999/09/23), source-RPM
1.3.8 (1999/07/13), source-RPM
1.3.7 (1999/06/06), source-RPM
1.3.6 (1999/05/17), source-RPM
1.2.2 (1998/04/28), tar file
1.0.3 (1998/02/11), tar file
1.0.2 (1998/01/29), email
0.99-38 (1997/01/21), diff, tar file, from DECUS
0.99-37 (1996/11/20), diff, tar file, from DECUS
0.99-36 (1996/11/12), diff
0.99-35 (1996/10/21), diff
0.99-34 (1996/09/03), diff, tar file, from DECUS
0.99-33 (1996/06/18), diff
0.99-32 (1996/06/09), diff
0.99-31 (1996/05/05), diff
0.99-30 (1996/04/29), tar file, from DECUS (used by Davis to build Lynx 2.4)
0.99-23 (1995/11/07), tar file, from DECUS
0.99-21 (1995/09/18), tar file, from DECUS
0.99-13 (1995/05/05), comp.editors posting
0.99-12 (1995/04/05), tar file, from DECUS
0.99-11 (1995/02/23), tar file, from dosemu

0.99-10 (1995/02/07), mentioned for LEO archive, dosemu

DOSEMU for Linux 94/11/19 ver. 0.53.35 unofficial pre-release

Sat Nov 19 1994

-       Latest terminal.c from John Davis.

Fri Nov 18 1994

-       Patch for PS/2 mice from Keith Parks <keithp@its.bt.co.uk>.
-       Slang code replacement in test. Set USE_SLANG in Makefile.
        Slang is a faster terminal control library than ncurses.
        Requires slang (amy.tch.harvard.edu in pub/slang , or tsx
        in the dosemu Development subdir). Many thanks to
        John Davis (davis@amy.tch.harvard.edu) for trying to
        win us over.
-       Patches to /init/parser.y for /etc/dosemu.users.

0.99-7 (1994/11/17), tar file, from DECUS
0.99-3 (1994/06/16) mentioned in Roger Young's list of finite element resources

0.99-2 (1994/05/19), tar file, from DECUS

S-Lang (pronounced ``sssslang'') is a powerful stack based language
interpreter with a C-like syntax which may be easily embedded into another
application, making it extensible.  Unlike a compiled language, S-Lang
functions cannot crash an application since S-Lang is interpreted.  If an
S-Lang procedure does crash an application, it should be regarded as either
a bug in the application or as a bug in S-Lang.  Therefor, not only does
S-Lang make an application extensible, it also provides a way to quickly
develop and debug the application in a safe and efficient manner.  Since
S-Lang resembles C, it is easy to recode S-Lang procedures in C if the need
arises.

0.98 (1994/04/12), tar file, from DECUS

0.95-3 (1993/11/29), tar file, from DECUS

S-Lang is a library that enables a C program to embed a C-like language
interpreter.  S-Lang is distributed with the GNU General public license and
it is also available for commercial licensing.  For details, send email to
davis@amy.tch.harvard.edu or write:

0.95-2 (1993/11/16), zip file, from DECUS
0.95 (1993/11/9), description of slang posted to alt.sources
0.94 (1993/06/12), note on foldoc

S-Lang

(language) A small but highly functional embedded interpreter. S-Lang was a stack-based postfix language resembling Forth and BC/DC with limited support for infix notation. Now it has a C-like infix syntax. Arrays, stings, integers, floating-point and autoloading are all suported. The editor JED embeds S-lang.

S-Lang is available under the GNU Library General Public License. It runs on MS-DOS, Unix, and VMS. Latest version: 0.94, as of 1993-06-12.

ftp://amy.tch.harvard.edu/.
E-mail: John E. Davis <davis@tch.harvard.edu>.

Documentation

The documentation for slang is sparse. This commentary has far more information. In a quick check of the Debian package libslang2-dev, there are a couple of documentation files, no manual pages:

$ zcat usr/share/doc/libslang2-dev/*.txt.gz|wc -
   8130   31570  256417 -

In contrast, ncurses's manual pages are about twice as large (counting only the manpage files from ncurses-doc in Debian 7):

  15837   88070  671692 -

The slang source-tree has more information:

$ wc cslang.txt slang.txt
  2796  16376 119686 cslang.txt
  6756  38300 266471 slang.txt
  9552  54676 386157 total

Those are translated from ".tm" files (text-files with markup):

 21722 104866 777816 total

However, much of that is stubs for undocumented stuff. Its undoc.tm lists 108 undocumented functions. Far more are simply not mentioned:

The slang 2.3.2 library has 711 public symbols, the documentation mentions 361 of those symbols.
In contrast, the ncurses6 library has 588 public symbols, with 120 from the sp-funcs category. These are all discussed in ncurses' documentation.

Source archives

Programs using either library:

Lynx (source-code since 1996-09-02)
Mutt (source-code since 1998-06-08)
Midnight Commander (source-code since 1998-02-27)

and the libraries:

ncurses (source-code since 1994-12-07)
slang (source-code since 2006-05-12)

and finally, the custom scripts/programs written for this discussion, with the caveat that the makefile makes assumptions, e.g., that some files such as picsmap.c have been copied into the current directory.

Followup

I began this page February 17, 2017, starting with notes which I had made in June 2011, for a proposed section in the ncurses FAQ named “Growth of the library filesize” (removed since it is now here).

This page was complete, except for the usual errata on December 12, 2017. Expanding that section in the ncurses FAQ is no longer necessary.

I may expand this section if there is anything interesting to report on how the material presented here is used.