http://invisible-island.net/personal/
Copyright © 2023 by Thomas E. Dickey

Copy-Wrongs – comments on plagiarism

Foreword

I began organizing information for this page in 2010, without completing it until now (August 2023). I have not run short of things to work on (and indeed, the scope of this page has grown). For instance, I have substantial work on pages for variations of the make program and the xterm meta-key. However, time passes, and I'm reminded that there is a limit to the time I have left.

Science Fairs

As a high school student, I developed projects for the school science fairs. I found competing in the high school and at the county level easy. Each year that I was in high school, I went on to the area fair. Doing that required placing first in a category in the county science fair. I did that with projects for these topics:

computing the eccentricity of an ellipse
measuring electrophoresis of slime molds
Pascal's pyramid
design, construction and testing a wind tunnel

Although my project received an award only once (the first time), I enjoyed the area fair because I did not merely visit it, but lived there for a couple of days, because it was far away from my home.

During my first visit, I stayed at the home of Dr. Rappleye (a biology professor). A few details were interesting:

it was a townhouse (not a detached dwelling)
they had potted cheese in the refrigerator
there was an recent issue of American Girl magazine on a table in the living room.

I asked if I might read that. It was probably the March 1966 issue, since the science fair was held on the weekend of April 16-18. To help with this page, I found its cover on Etsy, which looks familiar:

When I read the magazine, I found that it had stories (part of a contest). The first-place winner (a science fiction story) was submitted by a girl in California. But I had read that story before.

When I returned home, I went to the public library. They had a subscription to this magazine. I showed the magazine to the librarians, explaining that I was certain that I had read the story before. We searched through all of the science fiction books. It seemed the sort of thing that Frederic Brown would have written, and I was looking for that. However, when we found the story, it was by a less-known author:

“The Choice” by W. Hilton-Young
appeared in Groff Conklin's Omnibus of Science Fiction,
published in 1952.

It is very short. I found a copy here:

BEFORE WILLIAMS WENT into the future he bought a camera and a tape recording-machine and learned shorthand. That night, when all was ready, we made coffee and put out brandy and glasses against his return.
       “Good-bye,” I said. “Don’t stay too long.”
       “I won’t,” he answered.
       I watched him carefully, and he hardly flickered. He must have made a perfect landing on the very second he had taken off from. He seemed not a day older; we had expected he might spend several years away.
       “Well?”
       “Well,” said he, “let’s have some coffee.”
       I poured it out, hardly able to contain my impatience. As I gave it to him I said again, “Well?”
       “Well, the thing is, I can’t remember.”
       “Can’t remember? Not a thing?”
       He thought for a moment and answered sadly, “Not a thing.”
       “But your notes? The camera? The recording-machine?”
       The notebook was empty, the indicator of the camera rested at “1” where we had set it, the tape was not even loaded into the recording-machine.
       “But good heavens,” I protested, “why? How did it happen? Can you remember nothing at all?”
       “I can remember only one thing.”
       “What was that?”
       “I was shown everything, and I was given the choice whether I should remember it or not after I got back.”
       “And you chose not to? But what an extraordinary thing to—”
       “Isn’t it?” he said. “One can’t help wondering why.”

The story in the magazine was almost identical (substituting “tea” for “coffee”).

I wrote a letter to the publisher, explaining what I had found. I received no reply.

That was my initial first-hand encounter with plagiarism (as well as copyright violation). It was not the last.

A plagiarist presents another's ideas and the manner in which they are expressed as their own. Copyright only deals with the way something is presented.

Academia

As a college student, I ran into different issues.

Copying answers

Going from high school to college, I was uncertain what I should study. I was interested in physics, mathematics, and engineering. That was resolved by getting a scholarship at an engineering school. The same thing happened when I started graduate studies (though by then I had discarded physics).

Still, as an engineering student, I had some flexibility. Late in my freshman year, another student told me that there would be a course offered in numerical methods later that year (Fall 1970). I signed up for the course, and found that its prerequisite was the course on differential equations which I would be taking at the same time. To catch up, I worked through the problems in the latter's syllabus.

The other students in this class were mathematics majors, all juniors and seniors. The numerical methods course used Sokolnikoff and Redheffer's Mathematics of Physics and Modern Engineering, which I still have. We did not complete the book, but concentrated on a set of related topics. In particular, there was the vibrating string problem.

The instructor would leave the room during exams, to avoid disturbing us. During one of those, I glanced up, having noticed some movement, to see a senior craning his neck to look at what I was writing. I moved my paper to block that.

Surprised, I recall thinking to myself: “he must be stupid” not related to being caught, but rather “how is he going to know if my answer is better than what he would do for himself?”

Cheating in this way is a form of plagiarism, because the cheater uses another's work to (attempt to) get a better mark.

Personal gain is (usually) presumed to be the reason for plagiarism. However, stupidity plays a recurring role along with its partner, aggression.

Sharing grades

As an undergraduate engineering student, I wrote reports, both individually and in small groups (see for example my comments on terseness). As an engineer, both approaches are needed. Students need practice to be proficient with both.

Later, as a graduate student in electrical engineering, the (required) “internship in university teaching” course consisted of grading reports written by undergraduate engineering students. During 1975, Gary Leive and I graded reports by 87 students for the junior-level “analysis, synthesis and evaluation” course. All of the marks are on a scale up to ten (10).

Only a few students produced good reports. The rest required some judgement on our part. One assignment asked the student to identify advantages and likely problem areas to investigate in using a computer to control an automobile. One student wrote only that this had nothing to do with electrical engineering. I wrote zero (0), but the instructor overruled that. Just handing in a paper guaranteed six (6) points.

I kept that in mind with a later assignment to compute and report the results from a numerical solution to the Fourier heat equation. The students were given the formula to use in their programs, and the grid-size. All they had to do was write the short program and write a short report presenting the results.

The instructor specified individual reports so that the students would get useful experience in developing their own ideas and learning how to express themselves.

Reading and analyzing 87 reports takes several hours (even with two to share the work). It is important to use the consistent grading criteria. Partway through, I noticed a report that seemed familiar. Looking back through the graded reports, I found a match (the same phrases, etc). Gary and I changed tactics, setting aside reports which appeared similar. We found three clusters of reports which were problematic (totalling ten of the students).

After some discussion, we decided to split the grade up between the contributors. If a report merited 9 points, and there were three students in the cluster, each got 3 points.

Our decision this time was not overruled. Whether they worked together or copied from one was unknown. Either way, those students were not presenting their own ideas, in their own words.

At work...

Out in the world of paid work, versus gratis, I have little to report:

At the ITT Advanced Technology Center, I developed software using an IBM CMS front end, which provided the XEDIT editor. This editor was programmable via EXEC2 scripts.

CMS also provided FLIST, a file manager. I used this to explore the system, looking for useful programs which others may have written.

I came across a copy of one of IBM's XEDIT scripts, which had been modified:
- This was about 300 lines of EXEC2, implementing a command to change a range of lines.
- One line was added, to supply a default value for the number of lines to change.
- One line was removed — IBM's copyright notice.
On more than one occasion, I have encountered co-workers who claimed that some technical activity which I proposed, implemented, and reported on actually originated from them.

I do not work with them any more.

Unpaid work...

Outside the scope of paid work, things are more interesting. As outlined here, early in 1994, I began developing computer programs, distributed free of charge, on the Internet.

With no money involved, one might suppose that there is nothing to dispute. But there is that matter of attribution, which leads us back to plagiarism.

I have encountered too many disputes to go into depth for each. Instead, I have limited it to the more interesting stories.

wresize

As I described in my page on the ncurses license, I first dealt with ncurses to fix colors for my adding machine, and then with handling window resizing for my directory editor.

Raymond was reluctant to make the corrections for background color, and when I followed up with a configure script (replacing a crude script which Raymond had written) and then my changes to support SIGWINCH, he dug in his heels, so to speak.

Raymond's first mention of the imitation of wresize implied that he was not looking at my original, but the manual page was written in Raymond's style, easily recognizable as his own work. Given that, whether the imitation was a collaboration or not was a moot point (and dealt with, in due course).

However, that raised an issue which I had not previously encountered: the use of misattribution to discredit other developers, e.g., by stating that some other (inaccessible) person did the original work rather than the one who is being attacked. It is less straightforward than simple plagiarism, but serves the same purpose if aggression is the basis. Since then, I have seen this technique employed frequently.

Given the issue with manual pages, and this episode, I considered creating a webpage devoted to Raymond, since it appeared to me that half of his website contained dubious material. However, on closer examination I could only demonstrate problems in 19% of the site, and put that thought aside.

autoconf

This came shortly after the ncurses dispute, and has a similar story: a developer decided to emphasize his rejection of my proposed changes. To do this, he went beyond the dispute, plagiarizing something that I had developed (see discussion).

tparm, initial

The ncurses tparm function has some issues:

I started working with Eric and Zeyd early in 1995, beginning with the compiler-warning fixes in 1.8.9
At that point (seen in the source for 1.8.1), tparm was prototyped thus:
```
char *tparm(const char *string, ...);
```
It probably came from this change:
```
#### ncurses 0.7 -> ncurses 0.7.2 #### May 22, 1993 ####
...
* fixed lib_tparm.c to use stdarg.h (should run on suns now)
```
However, the function always copied 9 integer parameters from the stack. This is not the way to use <stdarg.h>.
Two years later, I modified tparm, as noted in the changes for ncurses 1.9;
```
### ncurses 1.8.9 -> 1.9
...
* rewrote varargs parsing in lib_tparm.c (to avoid referencing memory
  that may be out of bounds on the stack) -- Purify found this.
```
With that initial change, tparm determined the number of numeric parameters to expect, and retrieved just those from the stack.

In September 2002, I added checks for string parameters:

20000909
...
        + modify tparm to disallow arithmetic on strings, analyze the varargs
          list to read strings as strings and numbers as numbers.
        + modify tparm's internal function spop() to treat a null pointer as
          an empty string.

Reflecting on that, it improved ncurses but diverged from X/Open Curses which simply blessed the AT&T implementation from around 1985. That used a fixed-length parameter list:

char *tparm(char *, long, long, long, long, long, long, long, long, long);

To the extent that we discussed this in 1995, the consensus was that surely the standards people would keep up with the C language standard. Of course that did not happen.

Several years later, I helped review and update X/Open Curses,

Date: Tue, 5 Feb 2008 16:13:20 +0000
From: Geoff Clare <gwc@opengroup.org>
To: ogcurses@opengroup.org
Subject: tparm (was: Bug in CursesI5d2)

> From: Thomas Dickey
>
> On Thu, 24 Jan 2008, Geoff Clare wrote:
>
> > Problem:
> >
> > In this revision, the first parameter of tigetflag(), tigetnum() and
> > tigetstr() has changed to be const.  For consistency, the first
> > parameter of tparm() should get the same change.
> >
> > (In the preliminary draft the const was there, together with a change
> > from nine longs to an ellipsis; there were objections to the ellipsis,
> > resulting in the change back to nine longs, but I don't think there
> > was any objection to the const.)
>
> It's worth pointing out that the technical reason for the 9 longs is that
> some terminfo capabilities require those parameters to be strings rather
> than long/integer values.  (The reason for making that point of course is
> a concern that some platform may have a pointer that does not fit in a
> long).

I have been thinking about how best to address this issue of string
parameters.

As a minimum we should add some app usage text for tparm() explaining
that capabilities with string-valued parameters can only be used
in programming environments where converting a char * pointer to long
and back does not alter the value of the pointer.

I spot the following capabilities that have string parameters:

   pkey_key    (pfkey)
   pkey_local  (pfloc)
   pkey_plab   (pfxl)
   pkey_xmit   (pfx)
   plab_norm   (pln)

Are there any more?

If these are seldom used we could consider marking them obsolescent.

On the other hand, if they are widely used, and are felt to be
important enough, we should perhaps consider a more radical fix.
As previously discussed, we can't just change tparm() to have an
ellipsis in the prototype, as that would break existing conforming
apps.  What we could do instead is mark tparm() as obsolescent and
introduce a replacement function called, say, tiparm().  That way
existing apps would continue to work on XCurses Issue 5
implementations, and there would be a gradual migration path
from using tparm() to tiparm().

--
Geoff Clare <g.clare@opengroup.org>
The Open Group, Thames Tower, Station Road, Reading, RG1 1LX, England

adding the ncurses variation (actually a new entrypoint to the same code) as tiparm in January 2010:

20100109
...
        + add tiparm(), based on review of X/Open Curses Issue 7.

possibly reminded by a mention of X/Open Curses, Issue 7, as I noted recently:

The only new feature in issue 7 was tiparm, which was in ncurses a week after this answer. – Thomas Dickey

In the meantime, my implementation in ncurses was plagiarized twice:

by OpenBSD, in 1999. This copied from my initial implementation, which did not handle string parameters (see source code).

I noticed this issue in 2021, while investigating the relationship between NetBSD's tput and OpenBSD's tput. In writing this page (August 2023), I see that the tput.c file was updated recently:
```
Revision 1.28 / (download) - annotate - [select for diffs], Wed Feb 8 15:56:32 2023 UTC (6 months ago) by millert
Branch: MAIN
CVS Tags: OPENBSD_7_3_BASE, OPENBSD_7_3, HEAD
Changes since 1.27: +20 -50 lines
Diff to previous 1.27 (colored)

Use _nc_tparm_analyze() instead of trying to do it ourselves in process().
This fixes tput when used with a capability that uses push/pop.
OK gkoehler@
```
Since that (finally) uses the function which I wrote in 2000, the issue has been repaired. However, because OpenBSD has not upgraded their copy of ncurses for many years (and makes no improvements to it), this is still subject to the vulnerabilities which I addressed in April 2023.
by OpenQM, in 2005. Arguably that was merely a copyright violation because they copied more than a thousand lines of lib_tparm.c, removed the copyright notice and added a GPL license comment at the top, all without noticing that the library already did what they needed without modification.

However, no plausible explanation was provided for the bulk copying and removal of copyright notice (see initial discussion). That makes it more than a copyright violation, because OpenQM's developers intentionally misled others on the nature of the copying.

In contrast to OpenQM, when I noticed that Cristiano De Michele had copied from xterm in developing multi-gnome-terminal in 2002, he apologized and added a comment giving credit for the changes:

/*
 * Cristiano De Michele <demichel@na.infn.it>
 *
 * April 2002:
 * enhacements and bug fixings
 * for making zvt compliant with vttest
 * CREDITS:
 * Some code (see vt_clear_line_portion())
 * here has been taken from xterm, the popular terminal
 * emulator developed and maintained by
 * Thomas Dickey <dickey@herndon4.his.com>
 *
 * */

codeset-independent

See my note in the xterm FAQ for XITERM. Judging by the release notes, the unattributed copying was from by someone at IBM (another apparent blunder). This issue did not get much attention from Sun: the program (according to comments by Alan Coopersmith in the October 2002 and November 2003 threads) was used as the default “xterm” for some time.

The first newsgroup thread is more interesting to me for a different reason. It is where I first encountered Jörg Schilling. In writing this page, I checked to see whether his plagiarism of ncurses (see xtermcap) was perhaps motivated by some prior encounter. But I found no evidence of that. Our first encounter came in 2002, after his copying from ncurses. That is ironic given his attitude about Solaris versus every other system.

colored terminal? (October 2002)
configuring vi (January 2003)
Setting 132 columns in Solaris (November 2003)
VI Tmp File Too Large Error (November 2005)

XtInherit fixes

The issues with tparm overlap and bracket an occasion on which I was accused of plagiarism.

To recap, briefly:

The cygwin-xfree mailing list has some of this; the mail archives for XFree86 devel fill in some more pieces. While Cygwin/X CVS never found a home, but much of the work can be found in other repositories which currently are stored with Git, e.g., this one which can be browsed with cgit.
In October 2003, Harold Hunt posted to the XFree86 mailing list, demanding to be made a committer to XFree86 CVS because he found that spending a few hours every month preparing patches was wasting his time. If denied, he would separate Cygwin/X from XFree86.
I took a look at Harold's resume (which I still have). It told me that he was working on a master's degree, and listed some employment. However, the dates for the B.S. and prospective M.S. indicated that the employment was part-time. Further, he was probably interviewing for a full-time job. Given all of that, he was looking for an opportunity to improve his resume.

Harold's message to David Dawes put the latter into an awkward position, because Harold's tone was hostile, and according to David, private discussion had been worse:

Date: Sun, 26 Oct 2003 16:42:51 -0500
From: David Dawes <dawes@XFree86.Org>
To: devel@XFree86.Org
Subject: Re: Cygwin/XFree86 - Staying or leaving XFree86.org? [Was: Re:
        Cygwin/XFree86 Bugs?]

On Sun, Oct 26, 2003 at 12:37:29PM -0500, Harold L Hunt II wrote:
>Michel Dänzer wrote:
>>>Well, you know, XFree86's disregard for offers to help made by
>>>developers that have been with the project for over two years are
>>>certainly part of the problem.
>>
>> Err, this is about bug triage, which you can do just as well as
>> everybody else.
>
>No, this is about my submitting Cygwin-specific bugs that can't actually
>be assigned to me for committing them, even though I am the "expert" on
>Cygwin/XFree86.  This thread started to point out the hypocrisy of the
>situation and to see what the official response to this was.

When I discussed this with you privately a while ago all I got were
disrespectful and insulting responses.  Now there is more of the same.
As a volunteer myself, I don't have to take this type of attitude from
you or from anyone else.  And I won't.

David
--
David Dawes
Founder/committer/developer                     The XFree86 Project
www.XFree86.org/~dawes
_______________________________________________
Devel mailing list
Devel@XFree86.Org
http://XFree86.Org/mailman/listinfo/devel

I stepped in to point out that Harold's perception of the time involved was not shared, and got Harold's vulgar response.

While this was a mailing list with about 460 subscribers, most XFree86 contributors came and went, after providing small sets of changes. The XFree86 changelog lists 10564 changes. In that list, 15 developers account for half of the changes, while another 192 brings the total up to 90% (reaching down to halfway through the list of those with five changes).
Harold was not in the first 15, nor had he contributed as many changes as any of those whom I knew to be committers.
Most of the people who took part in the ensuing comments on this dispute were not (and have never been) contributors to either project.

Harold and I had never met, though he knew who I was, as seen in mid-2002:

Remember, we Cygwin/XFree86 folks are but mere packagers of XFree86 for
Cygwin; none of us work on xterm.  If you have problems with xterm, you
can take them up with the XFree86 project or with Thomas Dickey, the
primary developer of XFree86's xterm:

Harold knew who David Dawes was, having responded to messages forwarded by Alan Hourihane on the cygwin-xfree mailing list about XFree86 releases, e.g., early in 2003:

From alanh@fairlite.demon.co.uk  Tue Jan 14 15:14:00 2003
From: alanh@fairlite.demon.co.uk (Alan Hourihane)
Date: Tue, 14 Jan 2003 15:14:00 -0000
Subject: 4.3.0 status update
Message-ID: <20030114145100.GG824@fairlite.demon.co.uk>

This is what David Dawes wrote about the forthcoming 4.3.0 release.

Harold, Alex, and others. Can you take the time to really test the
current CVS and get patches in now.

Thanks.

Alan.

----- Forwarded message from David Dawes <dawes@XFree86.Org> -----

Here's a quick status update regarding the 4.3.0 release.
...

tparm, revisited

The most recent version of X/Open Curses (Issue 7) was published October 28, 2009. I happen to have a final copy from July of that year, because I participated in the review. It describes tiparm in detail (well beyond the information which I provided):

I had pointed out that the mixture of long and char* used by tparm was not really portable, and Geoff Clare proposed tiparm as a replacement. The X/Open Curses specifically says that the terminal capabilities which correspond to strings should not be used with tparm, and that tiparm should be used instead. The document's editor exceeded my expectations.

The problem comes about from other developers, e.g., those developing NetBSD's implementation of curses and terminfo. Some copy from ncurses, noting in the commits that they are implementing "SYSV curses".

Others document some things, but not others. I noticed this commit:

commit ba3282ab6d8d163de7b290a7f7c20749642e5e0b
Author: roy <roy@NetBSD.org>
Date:   Mon Oct 3 12:31:51 2011 +0000

    Correctly use ti_ instead of t_ for our extensions as per the man page.
    Replace vtparm with tiparm.
    tiparm is also non standard, but has been proposed at least.

and reading the corresponding manual page (terminfo.3) see neither a note that it was actually approved and published (three years earlier), or copied from ncurses. One of those possibilities should be documented.

Either way, there are pitfalls in maintaining these functions. In April 2023 (another ten years forward), I made security-relevant changes to tparm and tiparm in ncurses.

generally speaking, the problem is that in ncurses (and NetBSD), someone could manipulate the environment to cause the library to load a terminal description which causes it to malfunction.
The remedy which I chose ensures that the terminal capabilities actually came from the terminal database, as well as limiting the scenarios where string parameters could be used.

NetBSD terminfo is subject to the same problems, because it was based on the design from ncurses. Part of that is because the developer did not clearly understand the design from the outset, and therefore did not see the places where it might be improved.

Some programs have relied upon bypassing the terminal database by passing hard-coded strings to tparm. Some do worse, e.g., passing strings from tigetstr to printf. See for example Bug 904277 - sys-apps/openrc: broken with sys-libs/ncurses-6.4_p20230408 (shows [null] in place of color text code), color handling needs rewriting to use documented methods

Interestingly enough, the terminfo-related code in openrc was written in 2009 by the same person who developed the terminfo library in NetBSD. Although he was the main openrc developer (based on number of files changed, amount of change, etc.), his last involvement was in 2014. The second- and third-ranked developers are still active, but have made only minor changes to this area.
Regarding this topic, openrc has no change-log. Its AUTHORS file lists 101 contributors to that project. Writing in late August 2023, GitHub says there are 147 contributors, my git-stats script says there were 180 different committers. Reading through the list, there are a half dozen which could be combined due to variations in spelling.

janitors...

The dispute between Cygwin/X and XFree86 in 2003 involved a one-line entry in a changelog regarding the addition of 80 lines (more than half comments or whitespace) to a library with forty-four thousand lines of code and comments.

Our roles were reversed in mid-2010 when X.org developers copied more than two thousand lines of changes from work which I had done on luit, deleting the changelog which detailed the changes, and dismissing my work as janitorial cleanups. That was more than a little unfriendly, but something that I should have kept in mind when dealing with X.org developers, as you might understand by reading the mailing list archives a year before.

Up until that point, I was uncertain how I should mark my copyright on luit, because I had not calculated the percentage of change, etc. Typically I start copyright notices when I have changed twenty percent of a file. Some others use much lower thresholds, and then there are others who abuse copyright notices in other ways.

Keeping all of that in mind, I spent my time more productively by developing a new version of luit which would have no dependencies upon X.org sources. I copyrighted that, of course.

I started my page on change-logs at this point (September 2009), although it would have been topical earlier, e.g., in 2003:

I began adding notes for “reported by” in ncurses beginning in April 1996. Philippe Blain suggested doing this, and I concurred.
Keeping this information in the change-log is not uniformly regarded as necessary, nor uniformly reciprocated.
Later I amended my self-imposed guideline to refer to bug-tracking systems to solve a problem with users who were abusing my generosity.
Hostile and/or untruthful reports do not merit direct attribution.

xtermcap

After the encounters with Jörg Schilling in 2002 and 2003, there were others of note. For instance, in the 2004-2005 comp.os.solaris thread which led me document issues with Solaris Systems Programming, Schilling played a minor role. Some of it was pointless (for instance his praise of Bill Joy, whom he despised on other occasions as “Sun's first employee”), because of his pro-German bias. But for the most part, it consisted of attacks on Linux developers, e.g., those involved with cdrecord. Schilling was not the only person of interest. See for example:

Problem with Solaris 9 and mtr-0.58 (June 2004)
how to set argv command line args for ps (October 2005)

in one case giving poor advice, and in another blaming me for work done by others.

But seriously, I looked up “self-deprecating” thinking at first that Schilling had meant to say “self-discrediting” before recalling his bias.

In 2011, Yuri Pankov asked me for my response to Schilling's comments about the ncurses terminal database:

Date: Wed, 20 Jul 2011 16:31:51 +0400
From: Yuri Pankov <yuri.pankov@gmail.com>
To: Thomas Dickey <dickey@his.com>
Cc: Gordon Ross <gordon.w.ross@gmail.com>
Subject: Re: [illumos-Developer] A question about license for terminfo.src
        (Was: Re: webrev for 294, 1149)

On Wed, Jul 20, 2011 at 04:05:39AM -0400, Thomas Dickey wrote:
> On Tue, 19 Jul 2011, Gordon Ross wrote:
>
> > Thanks, Thomas.
> >
> > Yuri, perhaps you could find a suitable MIT copyright from some version of
> > this and put it in our THIRDPARTYLICENSE file or whatever?  (The file in
> > each source directory where we put the copyright and license text.)
> >
> > Thomas, can you point us to a suitable MIT copyright for this?
>
> There's lots of copies of the ncurses license in the source code...

Thanks Thomas,

and sorry for bothering you. One more question though, could you please
comment on the Joerg Schilling's reply about termcap/terminfo sources:

http://lists.illumos.org/pipermail/developer/2011-July/002697.html

Is it something we should care about?

TIA,
Yuri

Schilling was commenting about termcap only (he had neither interest or expertise in terminfo). I had been mechanically generating termcap from the ncurses terminfo source for several years after completing the work that Eric Raymond had begun in 1995. While Schilling said there were hundreds of errors, there actually were only a few problems in the translation. I fixed those later that week (July 23, 2011):

        + improve generation of termcap using tic/infocmp -C option, e.g.,
          to correspond with 4.2BSD (prompted by discussion with Yuri Pankov
          regarding Schilling's test program):
          + translate %02 and %03 to %2 and %3 respectively.
          + suppress string capabilities which use %s, not supported by tgoto
          + use \040 rather than \s
          + expand null characters as \200 rather than \0

That done, I was not finished. It occurred to me that there were different termcap implementations, and that they might not behave identically. I developed an application tctest and documented the different implementations which I knew about. I included Schilling's termcap variation, which turned out to be the least robust. There is a patch shown in my webpage on the topic which addresses this.

But there is more. Schilling's comment concludes with

To verify my statements:

-       get recent schily tools ftp://ftp.berlios.de/pub/schily/

-       compile

-       run the program "termcap" (a termcap compiler) with:

        termcap if=filename > newcap

filename is the name of a file to parse and to compile
newcap is the output file that contains comments with
hints to repaired bugs and to unrepairable bugs.

I investigated that, finding that it offered me no insight into termcap syntax. However, half of the program was copied from ncurses, as I noted here:

However, I did not incorporate any aspect of Schilling's test-program into
tctest.  It was not useful.

On the other hand, Schilling used ncurses in his test program.  It consists of
two files (cap.c and caplist.c).
Schilling copied the latter from ncurses 5.2 source code: include/Caps:

going on, to describe the copying.

Seeing that, I added Schilling to my plan for this page. There are other interesting stories to tell; I have only documented a few:

“termcap compiler” (its 2000 copyright date is consistent with the release date for ncurses 5.2).
patch, in which it turns out that Schilling based a day's work (twenty lines of code) on an early version of the GNU patch program while stating that it was really the original Larry Wall code.
tar, first published in 1997, claiming it was complete in 1983, while a change log from 1989 says the original source-archives (showing dates) was lost. Even with that, we have no sources predating the 1997 publication.

Here are some potential topics for which I have notes:

getline function, found in 4BSD sources, but having Schilling's 1986 copyright notice.
make program, to detail the changes made, etc., explain portability drawbacks.
cstyle script, likewise (changes and portability issues).

fixterms

Some developers do not bother with change-logs. Bram Mooleanaar's release notes for vim did not detail the changes, only telling what the entire program did. Since he first started using use source-archives late in 1999, this made it difficult to determine what was done, and who actually did the work. Reading the vim mailing lists and the comp.editors newsgroup helped.

That said, I cited Bram several times on my ncurses and xterm change-logs, for ideas and requests for features. He made one small change to xterm in 2004, making its usage message write to the standard output rather than to the standard error. That was not reciprocated (there are exactly two mentions of xterm with my name in the documentation, none in the commit logs, versus 51 cites in those two change-logs). Nor did we ever discuss vile (vi-like-emacs).

Some of our discussion did not lead directly to a fix or feature change in ncurses or xterm. On one occasion, Bram asked for more documentation:

Date: Fri, 11 Oct 2019 11:02:15 +0200
From: Bram Moolenaar <Bram@moolenaar.net>
To: dickey@his.com
Subject: Re: Prefixing a key when Meta is pressed

Thomas -

> > With patch 8.1.2135 I added the basics of the implementation.  It is not
> > enabled by default  That can be done with:
> >
> >     let &t_ti = "\<Esc>[>4;2m"
> >     let &t_te = "\<Esc>[>4;m"
> >     silent !echo
> >
> > The "silent !echo" is not needed when doing this on startup.
> > I can now map <C-Space> and a few other things works.
> > However, mapping <C-H> does the opposite: it maps the Backspace key and
> > not Ctrl-H.  That's a Vim problem.
>
> yes... the reason why I decided to make a table was to make it more
> apparent how the special keys would be handled.  Basically xterm's
> passing back the key event data (after some filtering/transforming
> which a table and the generating script would make more apparent).

I would like to see that table.

One thing I noticed was using CTRL-A (thus CTRL-Shift-a) result in
getting the CTRL and SHIFT modifiers and an upper case A.  That is a bit
strange, since the SHIFT is already included in the letter A.

> > I also added support in libvterm, so that I can do some debugging.
> > That's a bit tricky, since modifyOtherKeys has to be enabled in the Vim
> > instance, where the debugger runs, the Vim instance running in the
> > debugger, and the Vim instance running in a terminal in the debugged
> > Vim.  Yeah, that's confusing :-).
>
> mostly I use debug-logs (finding it simpler to study a log than
> single-step with gdb).

I do that sometimes, but it requires adding a lot of log statements (and
remembering to remove them again).

- Bram

--
Permission is granted to read this message out aloud on Kings Cross Road,
London, under the condition that the orator is properly dressed.

 /// Bram Moolenaar -- Bram@Moolenaar.net -- http://www.Moolenaar.net   \\\
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\  an exciting new programming language -- http://www.Zimbu.org        ///
 \\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

On this occasion, I wrote another webpage XTerm – “Other” Modified Keys about the origin of the features in xterm for passing modifier information of keys to an application. The page gives the table that Bram wanted to see, along with the script which I wrote to generate the tables.

Because Bram was in the process of getting libvterm to work with vim, he had course been exposed to Paul Evans' version of the facts. They are slightly skewed off. Referring to his webpage entitled “Fix Terminals – Please” I see several issues:

Some of this is unsubstantiated:

The way terminals encode modified keypresses is an extension to the original
methods used in the 1970s, and some older code cannot parse them correctly.
Press Ctrl-Up on a modern xterm and watch a program fail and
die in all sorts of interesting ways because they don't have real CSI parsers.

Some of it is heavily slanted:

Existing schemes usually use CSI ~ to encode special (i.e.
non-Unicode) keypresses.  XTerm and other terminals use the second CSI
parameter to pass encoding the set of modifiers in effect.

xterm implements both my original encoding and the variant which Paul Evans suggested.

I documented formatOtherKeys (a resource setting to choose either) in xterm's manual in 2008:

       formatOtherKeys (class FormatOtherKeys)
               Overrides the format of the escape sequence used to report
               modified keys with the modifyOtherKeys resource.

               0  send modified keys as parameters for function-key 27
                  (default).

               1  send modified keys as parameters for CSI u.

Some of it is simply omitted.

Paul Evans was not the first to implement his variant, nor even second. Apparently he was third. As usual, I used dated and published sources of information to arrive at that conclusion.

The variant presented as an innovation could be expressed as a single line of code. To make it clearer in xterm, I implemented that as a short function (shorter than the comment which Ralf Habacker wrote):

/*
* Function-key code 27 happens to not be used in the vt220-style encoding.
* xterm uses this to represent modified non-function-keys such as control/+ in
* the Sun/PC keyboard layout.  See the modifyOtherKeys resource in the manpage
* for more information.
*/
static Bool
modifyOtherKey(ANSI *reply, int input_char, unsigned modify_parm, int format_keys)
{
    Bool result = False;

    if (input_char >= 0) {
        reply->a_type = ANSI_CSI;
        if (format_keys) {
            APPEND_PARM(input_char);
            APPEND_PARM(modify_parm);
            reply->a_final = 'u';
        } else {
            APPEND_PARM(27);
            APPEND_PARM(modify_parm);
            APPEND_PARM(input_char);
            reply->a_final = '~';
        }

        result = True;
    }
    return result;
}

A few days later, Bram continued that email thread, commenting

Date: Sat, 02 Nov 2019 18:13:40 +0100
From: Bram Moolenaar <Bram@moolenaar.net>
To: Thomas Dickey <dickey@his.com>
Subject: Re: Prefixing a key when Meta is pressed

Thomas -
...
> > >
> > > Looks useful.  The background images are a bit distracting.
> ...
>
> here's something more complete:
>
> https://invisible-island.net/xterm/xterm.faq.html#xterm_modother
> https://invisible-island.net/xterm/modified-keys.html
>
> I have some work to do on the manpage and ctlseqs files to (try to)
> clarify some points.

Thanks, interesting history.  I didn't realize that Paul Evans was
already working on this stuff in 2008.  Also confirms that he has a bit
of a strange look on things, adding another key encoding, arguing it's
"much nicer".  And then forgetting to document how it works...  I
haven't verified it, but in the code it looks like his implementation
doesn't use Alt and uses the Esc prefix instead (which then causes the
timing problem).

- Bram

Not everyone conceded that I had documented the feature better than Evans. For example, in tmux #2669, Nicolas commented:

Nobody has really documented the keys well. xterm has a page on their variant
here https://invisible-island.net/xterm/modified-keys.html and the alternative
form is documented here http://www.leonerd.org.uk/hacks/fixterms/.

However, Bram and I occasionally discussed this area as well as others for ncurses and xterm. He started a new discussion in October 2022:

Date: Mon, 17 Oct 2022 17:11:49 +0100
From: Bram Moolenaar <Bram@moolenaar.net>
To: Thomas Dickey <dickey@his.com>
Subject: Kitty keyboard protocol

Thomas -

A few Vim users asked about supporting the kitty keyboard protocol.
It's like an advanced version of modifyOtherKeys.

Have you been contacted about it?  I wonder what your opinion is.

I thought it would be OK to enable with the TI and TE codes, like we do
for modifyOtherKeys.  But they suggested (or insisted) to first send a
request for whether the protocol is supported, and if the reply
indicates so only then enable it.  I told them that this is inconvenient
and rather enable it based on the termresponse value.  Or that they
implement modifyOtherKeys, since that is already more widely supported.
But that's where the discussion stopped.

One reason for the kitty protocol that was mentioned is supporting the
Cmd key on Mac.  I suppose xterm isn't used (much) on Mac, but perhaps
that keyboard can also be used on other systems.

There are few terminals and apps that support this kitty protocol, and
considering their attitute about using it I won't expect it will spread
quickly.  Wezterm is another one, first time I heard about it.

There was also mention that it was so great that the Esc key produces an
escape sequence, thus avoiding the need to have a timeout when a Esc is
received.  Somehow they didn't bring up 8-bit encoding (I also didn't
want to give away that obvious solution).  They appear to have a bit of
a one sided "my solution is the only good solution" view.

- Bram

--
My Go, this amn keyboar oesn't have a .

 /// Bram Moolenaar -- Bram@Moolenaar.net -- http://www.Moolenaar.net   \\\
///                                                                      \\\
\\\        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
 \\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

Calling it “like an advanced version of modifyOtherKeys” was stretching the point.

Date: Tue, 18 Oct 2022 12:02:03 +0100
From: Bram Moolenaar <Bram@moolenaar.net>
To: Thomas Dickey <dickey@his.com>
Subject: Re: Kitty keyboard protocol

Thomas -

Thanks for replying.

> > A few Vim users asked about supporting the kitty keyboard protocol.
> > It's like an advanced version of modifyOtherKeys.
> >
> > Have you been contacted about it?  I wonder what your opinion is.
>
> no.
>
> Looks like just a different way of encoding the same information.
> (add a few more modifiers, for the 12-fingered hackers...)
>
> Like modifyOtherKeys, with the same pitfall: the key-symbols that can
> be used depend on the keyboard configuration.

Yes, it has some small advantages that probably help few people.  It's
an interesting idea to send the Esc key as an escape sequence to avoid
ambiguity.  But as I explained to them, we have to deal with it anyway
because of how most terminals work.  And kitty only does it after
enabling, which may take a little while.

>From your reply I gather you have no plans to support the kitty
protocol, right?
...

agreeing with his conclusion

> >From your reply I gather you have no plans to support the kitty
> protocol, right?

right (it's not a priority - perhaps some user will offer a patch,
though it seems that most of those involved with stuff like this
aren't inclined to be constructive).

and we moved on to more productive issues of keyboard encoding.

Bram dropped the focus on kitty, widening it to keyboard protocols:

Date: Mon, 21 Nov 2022 00:04:00 +0000
From: Bram Moolenaar <Bram@moolenaar.net>
To: Thomas Dickey <dickey@his.com>
Subject: Re: Keyboard protocols

Thomas -

I recently made the libvterm copy that's in Vim work better, it now
produces the same codes with modifyOtherKeys level 2 as xterm, at least
for the keys I tried it with.  Makes it easier for me to debug the key
codes (I run gdb that starts Vim in a terminal window that's using
libvterm inside a running Vim).

> xterm does support the CSI u flavor, as I pointed out in

Vim understands both, and they contain the same information.  Thus it
doesn't really matter if the keys are encoded one way or the other.

Soon I'll add the kitty keyboard protocol, so I can try that out too.
I don't expect it to be complicated.  And we'll see how Paul Evans
reacts to that...

- Bram

--
>From "know your smileys":
 :-*    A big kiss!

 /// Bram Moolenaar -- Bram@Moolenaar.net -- http://www.Moolenaar.net   \\\
///                                                                      \\\
\\\        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
 \\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

His mention of Paul Evans was because Kovid's page on the topic borrows heavily from Paul Evans' page, amplifying all of its problems without providing any benefit.

color stack

In responding to one of Bram Moolenaar's questions about keyboard protocols, I noticed an unrelated problem on Kovid's website:

Date: Fri, 25 Nov 2022 04:20:17 -0500
From: Thomas Dickey <dickey@his.com>
To: Bram Moolenaar <Bram@moolenaar.net>
Cc: Thomas Dickey <dickey@his.com>
Subject: Re: Keyboard protocols

On Thu, Nov 24, 2022 at 10:29:06AM -0500, Thomas Dickey wrote:
> On Thu, Nov 24, 2022 at 02:12:38PM +0000, Bram Moolenaar wrote:
> >
> > Thomas -
> >
> > > > Anyway, for making Shift-Escape work I guess it needs to be added below
> > > > the "case XK_Return":
> > > >
> > > >                 case XK_Return:
> > > >                 case XK_Escape:
> > > >                 case XK_Tab:
> > > >                     result = (modify_parm != 0);
> > > >                     break;
> > >
> > > ok - added this to input.c and updated modify-keys.pl to match.
> >
> >
> > Thanks.  I assume the next version will have Shift-Esc working then.
> > Would that be version 377 ?
>
> yes ("soon", because I have fixes for a couple of bugs)
>
> I'm working on the newer query controls now, and will add in a few items
> from my backlog, but intend releasing 377 today or tomorrow.

...still on track for that (in testing this morning, found another hole in
one of my fixes).  Yesterday, I added two new sets of controls, to retrieve
the state information for the modified-keys feature and for the group of
"allowXXX" features.  The latter are (intentionally) set only via resource
or menu-action (and that only when SendEvents is off).

The "allowXXX" feature is best done using names, so that pointed me to
implementing it with OSC.  Since several programs have their own quirks
with OSC's, I did a quick check for conflicting codes, and came across
this:

https://web.archive.org/web/20211027194253/https://sw.kovidgoyal.net/kitty/color-stack/

which is untrue.  On investigating, I see that Kovid implemented a push/pop
for the current colors in 2018 using DCS, and then changed it to OSC in 2021,
at the time that he made this webpage, with the comment about xterm.

I added the sgr-stack feature also in 2018, which pushes/pops colors, but
intentionally limited that to the current colors, since it pushes/pops only
the attributes specified.

That's not useful for managing the 256-color palettes, and (you've probably
also seen) some users prefer one version of Solarized on the command line,
and a vim theme in the editor.  I resolved that in 2020 with another control,
for managing the color palettes.  In that, the current colors (which Kovid
refers to as the dynamic colors) came along for consistency.

Since he doesn't document things systematically, I only noticed this during
this week.  The keyboard protocol page has similar issues, but at the moment,
I don't see a need to address either on my webpages.

> > The changes to also support the kitty keyboard protocol are mostly done.
> > Now awaiting users to complain about any problems.  I also made some
> > improvements for how modifyOtherKeys is used and updated the help.

--
Thomas E. Dickey <dickey@invisible-island.net>
https://invisible-island.net
ftp://ftp.invisible-island.net

Bram was not very interested in pursuing this (since his interest in color schemes was solely for vim). However, since the page stated

In July 2020, after several years, XTerm copied this protocol extension,
without acknowledgement, and using incompatible escape codes (XTPUSHCOLORS,
XTPOPCOLORS, XTREPORTCOLORS).  And they decided to save not just the dynamic
colors but the entire ANSI color table.  In the interests of promoting
interoperability, kitty added support for XTerm’s escape codes as well, and
changed this extension to also save/restore the entire ANSI color table.

I added it to my plan for this page.

Readers knowledgeable about the technical area may find it bizarre, but those familiar with Kovid Goyal likely will have a different reaction.

In short, Kovid accused me of plagiarism. The facts do not support the accusation.

Kovid's documentation is rather weak (i.e., sweeping claims but no analysis), so referring to his Git repository is the only way to see what he might be talking about.
The page mentions two escape sequences
```
<ESC>]30001<ESC>\  # push onto stack
<ESC>]30101<ESC>\  # pop from stack
```
The “<ESC>]” pairs are the 7-bit form of OSC, one of the control sequence prefixes documented in ECMA-48. XTerm implements several (see documentation), relying upon an informal convention using a numeric parameter with an associated string value.

If the control has only numeric parameters, it is more suitable to CSI (see documentation).

The 30101 first appeared in kitty/parser.c, with

commit 6f7ea490081a594fafba190f315c7ea2579c91fb
Author: Kovid Goyal <kovid@kovidgoyal.net>
Date:   Fri Jun 15 15:21:46 2018 +0530

    Switch to high number OSC codes instead of DCS string codes for dynamic color push/pop

At that point, Kovid documented the feature thus:

Saving and restoring the default foreground/background/selection/cursor colors
---------------------------------------------------------------------------------

It is often useful for a full screen application that has its own color themes
to set the default foreground, background, selection and cursor colors. This
allows for various performance optimizations when drawing the screen. The
problem is that if the user previously used the escape codes to change these
colors herself, then running the full screen application will overwrite her
changes even after it exits. To avoid this, kitty introduces a new pair of
*OSC* escape codes to push and pop the current color values from a stack::

    <ESC>]30001<ESC>\  # push onto stack
    <ESC>]30101<ESC>\  # pop from stack

These escape codes save/restore the so called *dynamic colors*, default
background, default foreground, selection background, selection foreground and
cursor color.

The DCS controls are only a little older than that commit:

commit 82f9f002bdf6802347134f97beb023da887989dd
Author: Kovid Goyal <kovid@kovidgoyal.net>
Date:   Thu Jun 14 23:25:13 2018 +0530

    diff kitten: Fix default foreground/background colors not being restored when kitten quits. Fixes #637

That is, if I had copied Kovid's feature, this was only two years later (hint: “several” is always more than two).

Kovid's accusation currently appears in docs/color-stack.rst, but first appeared in docs/protocol-extensions.rst in this commit:

commit 5f8dee8384dc50e59b7394157d5c5d352c4dc836
Author: Kovid Goyal <kovid@kovidgoyal.net>
Date:   Mon Dec 21 21:39:05 2020 +0530

    Add support for the color settings stack that XTerm copied from us without acknowledgement and decided to use incompatible escape codes for.

    Completely in keeping with that project's past behavior.
    See https://github.com/kovidgoyal/kitty/issues/879

    XTerm announcement:
    https://www.mail-archive.com/xorg@lists.x.org/msg06419.html

alluding to patch #357 (July 5, 2020).

That commit did not merely “Add support” but (reading the changes for the alternate screen) replaced the documented OSC-sequences with those which Kovid copied from xterm.

Immediately before that, Kovid made this change:

commit e97f1a431023fefd74c746f9a38e87a21eb17e5c
Author: Kovid Goyal <kovid@kovidgoyal.net>
Date:   Mon Dec 21 19:38:03 2020 +0530

    Have the save/restore colors escape codes also save restore the ANSI color table

That is, not only did Kovid accuse me of plagiarism, but at the same time he did what he accused me of doing.

Stacks in programs predate my use of them, as well as likely preceding Kovid's existence. I have designed and implemented more than one control sequence for xterm which uses a stack:

title-stack in patch #252 (December 7, 2009), improving security.
SGR-stack in patch #334 (August 12, 2018), starting from a user request.
color-stack in patch #357 (July 5, 2020), improving usability.

Occasionally someone suggests (I am one of those) that the alternate screen feature of xterm could be done better as a stack. There are pros and cons about that, which would be a distraction from this topic.

In the instance of the color-stack, the motivation went back several years, before Kovid began developing kitty in October 2016.

The need for a feature for switching palettes came about with the popularization of the solarized theme after its inception in February 2011.
The problem with solarized (and any non-default color scheme) was that there was no convenient means to switch between different palettes, e.g., one for vim and one for command-lines.

Over the years, I made a few notes for a time when I might work on that problem.

Here are notes from my to-do list for xterm that dealt with switching palettes. The most recent one (200629, or June 29, 2020) mentions a test-script which I wrote for testing the color-stack feature. Test-scripts for xterm which I have not made generally available are named “vxt-NAME” — this is one that I released, renaming it to palettes.pl.

200629
**      while writing vxt-palettes, found that sometimes the response for
        OSC 10/11 did nothing for the former, putting both replies in the
        latter.  That's because xterm buffers the response, and will not
        return the whole result until some time has passed.  I can't force
        a flush just for this special case, but can do this:
        + force it to read until the reply has a string-terminator, or
        + do bulk-query (and force the read to wait for all replies).
...
200612
**      palette push/pop should include both Tcolors and Acolors, using
        a mode to select one or both of those.
...
200410
***     add palette-stack (or pool) for switching palettes
        Do this with code 12 of the title-stack.
...
120605
**      xterm would be much faster at palette updates if I could mark some
        updates as pending-repaint, i.e., the operation will force a repaint
        of the screen, but (like refresh in wgetch) do not do the repaint until
        some operation slows things down.  It seems that all of the SGR
        operations could be merged into this pending-repaint, so doparse
        could keep postponing the repaint as long as it has an SGR or any of
        the substates (CSI, semicolon, digit) which make that up.
...
100724
        it would be nice to be able to reset the palette to a default and/or
        save a new set of default values.

Seeing that the to-do items were rather old, most readers may understand that I have more ideas than time (which is much better than the opposite extreme), and find no need to copy ideas from others. Of course that is not the case with others, hence this page.

To support my development, I do have copies of the source for hundreds of programs. Originally that meant tar-files (I see 2735 in that folder), CDROMs and DVDs (a few hundred), but currently most come from Git repositories (and a few from other systems such as Mercurial and CVS). The source for kitty is in a directory with 187 Git clones, making it simple to update those with a script. When I discussed keyboard protocol, and in writing this page, I refer to that offline copy.

But unless I have a specific question to answer using those sources, I do not study them. I am too busy. Again, others are not.
Since I began with a wish-list item in 2010, it is clear that seeing solarized was not the first time I considered the problem. But having many, many users doing something with the color palette got my attention.

Kovid referred to these features in XTerm Control Sequences. I gave them names, which is simpler than typing the escape sequence, and encoded them using CSI because that fits with the convention used by other controls:

Kovid ignored these related features (from SGR-stack):

Kovid probably ignored the SGR-stack features because they are more complicated than the color-stack features. If he had examined them, he would have undoubtedly added another paragraph accusing me of plagiarism for the special case where it saves and restores the foreground and background color.

The SGR-stack feature was initiated by Dan Thompson in three emails starting June 9, 2018:

Date: Sat, 9 Jun 2018 17:28:27 +0000
From: Dan Thompson <dan@thosethompsons.org>
To: Thomas Dickey <dickey@invisible-island.net>
Subject: xterm: SGR control sequences: method to do push/pop?

I've got a question about Control Sequences, specifically regarding Select Graphics Rendition (SGR).

I've been writing code that emits text that contains SGR control sequences to colorize it. However, I've found that my text is not easily composable--that is, I may have a format string like "The name is: %s, the position is: %s", and then I want to sprintf some inserts into that. Then that whole string I may want to insert into another, and so forth. The problem is that to have SGR sequences in that one has to have sort of a global awareness of content and associated coloring for that to work out.

What I'd *like* to do is have some sort of push/pop mechanism for the SGR settings, so that I could have strings like:

    <fgBlack>   : CSI 3 0 m
    <fgGreen>   : CSI 3 2 m
    <fgYellow>  : CSI 3 3 m
    <fgMagenta> : CSI 3 5 m
    <fgWhite>   : CSI 3 7 m
    <bgBlack>   : CSI 4 0 m
    <bgYellow>  : CSI 4 3 m
    <bgWhite>   : CSI 4 7 m
    <pushSgr>   : CSI ??? m
    <popSgr>    : CSI ??? m

    name = "Dan <pushSgr><fgBlack><bgWhite>Thompson<popSgr>"
    position = "<pushSgr><fgMagenta><bgYellow>Developer at Large<popSgr>"
    line1 = "<pushSgr><bgBlack><fgWhite>The name is: %s, the position is: %s<popSgr>"

And then be able to do printf(line1, name, position), and have it all work out.

Without a push/pop mechanism, composing strings like that cannot be done that way at all, unless you do your own custom rendering pass to handle the pushes and pops.

He followed up with a patch on June 14, and an update on June 22. Just saving and restoring all of the video attributes was not interesting to me, so I modified it to allow for selecting the attributes to be saved by parameters. If this had come after the color-stack feature, I probably would have used consistent parameters for the two features. But because of my changes, it is possible to save and restore the foreground and background colors without the other attributes, like this (using the same convention as Kovid's page):

<ESC>[#30;31{        # push foreground and background colors
<ESC>[#30;31}        # pop foreground and background colors

Thompson's patch made the feature SGR 66 and 67, which (ignoring subparameters) did not fit in with my improvement. I used curly braces for the final characters to make the feature easier to read. Thompson did not like the curly braces because he found it inconvenient in C#, although the Microsoft documentation shows it would be simple. At the time, that documentation was not readily available, so I added aliases for the feature:

XTPUSHSGR (alternate)
XTPOPSGR (alternate)

For reference, here are sample scripts which I wrote to test and demonstrate the three stack-related features of xterm:

and here is an illustration of solarized and other themes using the color-stacks feature:

Unresolved issues

Not all of these issues are in the past.

Misattribution

This is still (after more than twenty-five years) an issue as new developers come to invent this method of detracting from others' work. After all, there is no point (for some) in improving one's resume unless they can do this by taking it away from someone else.

You can see this issue by the way changes are documented – or not. Several examples are mentioned in my page on bracketed-paste.

Continuing the previous section, Kovid is among several developers who practice this methodology. e.g., this change referring to the xterm title-stack feature:

commit 3067103b188c55d594918d1525d397cb523f8fea
Author: Kovid Goyal <kovid@kovidgoyal.net>
Date:   Wed Aug 29 09:56:52 2018 +0530

    Implement window title stack

    Used by new versions of vim

This was one of several attempts by Kovid to make kitty imitate xterm well enough that Bram would be forced to do the remaining work to accommodate kitty as a slight variation on xterm. Bram was neither convinced, nor treated well by Kovid in their encounters, e.g., this:

After 9.0.1080, &tgc doesn't work in kitty #11729

As one might guess, I have also encountered Kovid. At one point, Kovid claimed to acknowledge my work:

commit 952aa7ad4ada9ec46fe023ac7cbedfc2b3ab518c
Author: Kovid Goyal <kovid@kovidgoyal.net>
Date:   Tue May 23 20:27:51 2017 +0530

    A tribute to Thomas E. Dickey

but later (silently) removed that

commit f87d62571bcf58f0b72c0181d340eccebded026e
Author: Kovid Goyal <kovid@kovidgoyal.net>
Date:   Fri Feb 23 09:13:20 2018 +0530

    Add links to relevant parts of the README in the major features list

shortly after this discussion on bug-ncurses:

   From:  Kovid Goyal <kovid@kovidgoyal.net>
Subject:  Request for new terminfo capability (DECCARA extended)
   Date:  Wed, 7 Feb 2018 08:23:48 +0530

As the developer of a library supporting many types of terminals, I will not spend time writing specialized code to support one. Kovid's interactions with other developers generally are exploitative. When thwarted, he responds.

Kovid came back, having arranged with someone else to act as his agent:

   From:  Igor Urazov
Subject:  Request for inclusion of new xterm-kitty terminfo
   Date:  Thu, 6 Sep 2018 16:13:16 +0300

Notwithstanding his comments about the FAQ, Kovid had only read the parts that he was interested in. This one is a good starting point:

Why not just use TERM set to “xterm”?

Change Logs

By sheer coincidence, I found that revisiting mawk 1.9.9.6 after seven years has some relevance to this page's discussion of change-logs.

Mawk 1.9.9.6's README has some rather hostile remarks pointed at me:

In my absence, there have been other developers that produced mawk 1.3.4-xxx.
I started from 1.3.3 and there is no code from the 1.3.4 developers in
this mawk, because their work either did not address my concerns or
inadequately addressed my concerns or, in some cases,
was wrong.  I did look at the
bug reports and fixed those that applied to 1.3.3

While I was aware that the comment about Mike's concerns was misleading (and addressed that in 2016), I did not notice that the disclaimer that “there is no code from the 1.3.4 developers” was also untruthful. I discovered that in July 2023 while analyzing this bug report:

20230725
        + use da_string more consistently in dumps.
        + improve scanner to some type-checks of arrays versus scalars by
          deferring this into the runtime execution (report by Rajeev V Pillai).

Rajaeev gave an example in email, and mentioned some variations. I made some modifications, e.g., replacing an environment variable with a literal string to make it suitable for regression testing.

# Subject: Re: mawk 20230525: var-type confusion
# On Sun, Jun 04, 2023 at 07:27:26AM +0000, Rajeev V. Pillai wrote:
function xdelete(a,     i) {
        for (i = 1; i <= length(a); i++) {
                printf "deleting %d:%d %s\n", i, length(a), a[i]
                delete a[i]
        }
        #for (i in a)
        #       delete a[i]
}

BEGIN {
        print split("/usr/sbin:/sbin:/usr/bin:/bin:/usr/games", a, /:/)
        xdelete(a)
        print length(a)
}

I tested Rajeev's example with gawk, original-awk, mawk 1.3.3, 1.3.4, and 1.9.9.6, finding that it failed with both 1.3.3 and 1.3.4 (so the problem was not a regression due to my changes). The failure occurred at the parsing stage:

mawk: raj_array.awk: line 5: illegal reference to local variable a
mawk: raj_array.awk: line 6: illegal reference to local variable a
mawk: raj_array.awk: line 14: type error in arg(1) in call to xdelete
mawk: raj_array.awk: line 15: illegal reference to array a

However, since it worked for 1.9.9.6, I next examined the output from mawk's assembly-language dump.

As expected, there were some differences, though the numeric-offsets between the two dumps make it hard to read. Part of that was due to my formatting change to add a column of dots (separating the offsets from the mnemonics). To help eliminate that nuisance, I made a copy of 1.9.9.6, and edited da.c to make this cosmetic change.

At that point, I noticed that Mike's version of that file had a function function named visible_string, while mine had the same function (except for that line) but named da_string. Curious about that, I checked my source archive and found that I had written that function in 2009 as part of my fixes to allow RS to contain an embedded null. It certainly was not in mawk 1.3.3 (disagreeing with the disclaimer in the README file).

That gave me something to think about. In the meantime, I was curious how Mike had solved the problem of the ambiguous forward reference to the length function. I proceeded by applying changes from 1.3.4 to this copy of 1.9.9.6, to reduce the total difference to review. In that process, I found additional borrowing from my work. Reviewing the result, there were four commits in which I noticed the borrowing, and noted it in the check-in message. Now these are fairly small, but undeniable borrowing from my work which Mike said he did not do.

There also was the comment header, which says

Mawk is distributed without warranty under the terms of
the GNU General Public License, version 3, 2007.

If you import elements of this code into another product,
you agree to not name that product mawk.

Actually that is contradictory, but I get the point. Mike was again addressing me.

If it were not for that README file, Mike would get better treatment. However, I have my own policy regarding that, dating back to 2009. In this case, I adapted Mike's changes.

Continuing, I pinpointed the relevant code, and could see how to adapt the idea. Considering the situation, I decided to acknowledge the idea on this page, explaining why a direct borrowing from Mike's version was unsuitable. If Mike wants to discuss this with me, he will have to be polite and accurate.

The license is based on copyright, which essentially limits it to prohibiting literal borrowing. It is not a patent, which would protect the algorithm. I used the algorithm, and eliminated literal borrowing.

Starting from my heavily modified copy of 1.9.9.6,
I made a bug-fix to allow it to fail gracefully in my regression tests,
I rewrote all of the comments in the adapted code,
renamed the variables (to make them actually understandable),
fixed a shift/reduce conflict which Mike's changes introduced into parse.y,
made the switch statements consistent,
applied my changes back to the modified copy, for regression testing, and
added test-cases to my regression tests to validate the change.

I preferred not making the bug-fix, but without that, mawk 1.9.9.6 was not able to get to the test-cases which I added to validate the new feature.

While developing the test-cases, I found another case of ambiguous length which Mike's change did not handle, and spent some time working on a solution for that.

In the meantime, I had targeted completing this page in August 2023 for various reasons. Therefore, I put aside the work on mawk, expecting to resume on mawk after some maintenance on xterm.