Subject: Escape sequences for xterm?

From: [email-masked] (Curly++)
Date: Fri, 3 Mar 2000 15:08:59 -0500

I think what I need is a listing of xterm escape codes.  All the 
listings I've found are horribly out of date. 

I've done a few extensive web crawls but haven't been able to 
find anything up-to-date except the source code... yeah, I know, 
but it's self-documenting!

Maybe I'm going about this all wrong, so let me tell you what I'm 
doing, you can tell me if I'm stupidly ignoring an obvious and 
simple solution.

I'm fiddling with the source for xterm.  I need to monitor the 
user's interaction with some applications so as to divert some 
of the inputs to another app.  Legacy stuff, this is a kludge 
to bridge the old and the new until the old can be fully phased 
out.  I'm writing scripts with `expect` to wait for the user 
to fill in a form, then take an image of the screen and pass it 
to another app.  I can't capture the data coming back, the 
screen is updated in tiny bits sort-of like what curses does. 

I've added an escape sequence to xterm so the expect script can 
be triggered by the few predictable strings and use `send_user` 
to trigger a screen dump to an external file.  I'd like to use a 
"correct" sequence and submit this as a patch to the xterm 
maintainer.  This seems to me a general solution to a set of 
problems that turn up occassionaly. 

-- 
Oisin  "Curly++"  Curtin                     [email-masked]
Surface Liaison, Minetown Digger                    Send no SPAM.
                                http://pages.infinit.net/curlypp/

---

From: [email-masked] (Martin Ramsch)
Date: 3 Mar 2000 22:51:21 +0100

On Fri, 3 Mar 2000 15:08:59 -0500,
> listings I've found are horribly out of date.

Did you also have a look at the file "ctlseqs.ms" which is part of
every xterm packages?

To convert this file into other formats use for example
  groff -ms -t -Tps ctlseqs.ms >ctlseqs.ps       for Postscript output
  groff -ms -t -Tlj4 ctlseqs.ms >ctlseqs.lj4     for LaserJet 4 print file
  groff -ms -t -Tdvi ctlseqs.ms >ctlseqs.dvi     for TeX dvi output
  groff -ms -t -Tlatin1 ctlseqs.ms >ctlseqs.txt  for text output

Regards,
  Martin
-- 
Martin Ramsch <[email-masked]> <URL: http://home.pages.de/~ramsch/ >
PGP KeyID=0xE8EF4F75 FiPr=5244 5EF3 B0B1 3826  E4EC 8058 7B31 3AD7

---

From: "T.E.Dickey" <[email-masked]>
Date: Sat, 04 Mar 2000 11:37:52 GMT

> Did you also have a look at the file "ctlseqs.ms" which is part of

Technically, ctlseqs.ms was moved to the doc directory tree some time
ago - depending on how you get xterm, you may not see it - I bundle it
in my xterm tarballs for convenience.

The XFree86 xterm supports ANSI color and VT220 emulation
There's an faq at
	http://www.clark.net/pub/dickey/xterm/xterm.faq.html
	ftp://ftp.clark.net/pub/dickey/xterm

(altavista usually shows my webpage - though I don't search for that ;-)

-- 
Thomas E. Dickey
[email-masked]
http://www.clark.net/pub/dickey

---

From: "T.E.Dickey" <[email-masked]>
Date: Sat, 04 Mar 2000 14:39:17 GMT

> problems that turn up occassionaly. 

While it's useful, this is generally frowned upon as a security problem.
I don't think it'll get much general acceptance.

---

From: Paul Williams <[email-masked]>
Date: Sat, 04 Mar 2000 18:33:37 +0000

> I don't think it'll get much general acceptance.

It is possible for a host computer to grab the contents of the screen
of a Digital VT420. Howeve, xterm doesn't emulate a VT420 (yet?)

---

From: [email-masked] (Curly++)
Date: Sun, 5 Mar 2000 15:32:03 -0500

>   groff -ms -t -Tlatin1 ctlseqs.ms >ctlseqs.txt  for text output

Ah!  Thanks, Martin.  That's the answer all right!

I'm not a gruff person, er, I mean I'm not a groff person so the 
.ms didn't mean anthing to me.  I'll suggest that your answer be 
added to the FAQ for all us ignoramuses. :-)

---

From: [email-masked] (Curly++)
Date: Sun, 5 Mar 2000 15:39:42 -0500

> ago - depending on how you get xterm, you may not see it - I bundle it

I must have an old copy then. 

AFAIK, I did check the lastest FAQ at clark.  Can MR's reply be 
added so "us unwashed masses" can cut'n'paste the right command 
to see it?  TIA, either way.

---

From: [email-masked] (Curly++)
Date: Sun, 5 Mar 2000 16:15:02 -0500

> I don't think it'll get much general acceptance.

DISCLAIMER: I'm not a security expert and I don't play one on TV.

You're referring to the log problem?  I haven't read all about 
it, but I gather it's not logging per se, but the fact that the 
destination file is uncontrolled that causes a problem.  

I plan to make screen-scraping secure by allowing the user only 
limited control of the destination filename and ignoring the 
request if the file already exists.  That means the screen-scrape
will never, for example, add a line to ~/.profile.  This forces 
the external script to delete files, but then the script needs 
much less logic to read the image that way. 

I'm not sure, but I think the solution to the logging problem is 
similar: restrict the name and location of the file to something 
less than anywhere/anything and use more elaborate fstat code in 
place of the access tests.  Maybe even put a magic cookie at the 
top of new logs and never append to an existing log if it doesn't 
have a cookie on top.  A cookie is a lot of extra code though, I 
wouldn't do it unless the real security experts say it's needed. 

BTW, I have to add this...
My partner is not used to seeing me grinning from ear to ear and 
muttering under my breath "Wow!  That's neat!" and similar things. 
This code is an absolute _pleasure_ to work with.

---

From: [email-masked] (Curly++)
Date: Sun, 5 Mar 2000 16:32:52 -0500

> of a Digital VT420. Howeve, xterm doesn't emulate a VT420 (yet?)

But doesn't that go back to the remote application?  I think 
`expect` could divert the output, but having dealt with timing 
issues before, I feel more comfortable if the data *cannot* 
go out the wrong way no matter how hard it tries.

---

From: "T.E.Dickey" <[email-masked]>
Date: Sun, 05 Mar 2000 23:55:17 GMT

> to see it?  TIA, either way. 

maybe (I've been more busy fixing bugs in various programs recently than
updating webpages).

---

From: [email-masked] (Curly++)
Date: Mon, 6 Mar 2000 12:37:00 -0500

> DISCLAIMER: I'm not a security expert and I don't play one on TV.

Geez, I'll never get that TV job.  <dramatic sigh>

I went back to look at Morten Welinder and Branden Robinson's 
postings to bugtraq.  Ya now, I don't even have to think to see 
how *machines* could get confused splitting the data stream 
between the local screen-image and the remote host.  But Morten 
Welinder clearly is describing the same kind of timing issue, 
except the failure is in human ethical subroutines.  I didn't see
it until I'd read it three times.  <sigh> Such is the life of an 
optimist. <grin>

I'm off to carefully read (at least three times!) the proposed 
logging patch.

---

From: [email-masked] (Jeffrey Altman)
Date: 6 Mar 2000 22:29:06 GMT

In article <[email-masked]>,
Curly++ <[email-masked]> wrote:
: In comp.terminals, Curly++ proclaimed:
:  
: > DISCLAIMER: I'm not a security expert and I don't play one on TV.
: 
: Geez, I'll never get that TV job.  <dramatic sigh>
: 
: I went back to look at Morten Welinder and Branden Robinson's 
: postings to bugtraq.  Ya now, I don't even have to think to see 
: how *machines* could get confused splitting the data stream 
: between the local screen-image and the remote host.  But Morten 
: Welinder clearly is describing the same kind of timing issue, 
: except the failure is in human ethical subroutines.  I didn't see
: it until I'd read it three times.  <sigh> Such is the life of an 
: optimist. <grin>
: 
: I'm off to carefully read (at least three times!) the proposed 
: logging patch. 

There is no safe way for a host to specify a file name that should
be used for logging, or printing, or any other kind of output.  If
host automated logging is the be provided it MUST be:

 . authorized by the client

 . to a file specified by the client

This should not be done via a standard control sequence.  If it is 
going to be done it should be performed via a private sequence (preferably
not a '?' sequence) or via an APC.

    Jeffrey Altman * Sr.Software Designer * Kermit-95 for Win32 and OS/2
                 The Kermit Project * Columbia University
              612 West 115th St #716 * New York, NY * 10025
  http://www.kermit-project.org/k95.html * [email-masked]

---

From: [email-masked] (Curly++)
Date: Mon, 6 Mar 2000 22:10:52 -0500

> be used for logging, or printing, or any other kind of output.  If

It's a connundrum, all right.  I started this thinking it would be a 
breeze.  Making the part I needed was simple and straightforward. 
Fitting it into the configurator was a bit more work.  But making it 
secure enough to distribute... this is tough!

>  . authorized by the client

Yes.  Authorization may be implicit, just as you "authorize" someone 
to modify your log files by connectiong to your anonymous FTP server, 
so too you might provide a very limited authorization to write to a 
file within a carefully proscribed namespace. 

>  . to a file specified by the client

I've only really considered *nix for building a name.  But I 
suspect this can be easily addapted to other OSs.

I had hoped I could use the code that opens the log in the patch 
posted to bugtraq, but the method seems broken to me.  Also, I 
can't see how it deals with the symlink problem, but I haven't 
had time to play with it yet.  It is broken in terms of the 
forced naming.  First, it allows the file to be anywhere because 
it opens it in the current directory.  I think it should be 
restricted to a selection between the user's home directory and 
the system-wide tmp directory.  Second, using the time to create 
a "unique" name is just wrong.  A user starting two sessions from
the same shell script will have a problem, but can deal with it 
by adding "sleep" to slow down that expensive, fast computer.  
However, if it's two different users clicking at the same time, 
the user cannot predict or prevent the collision.  It would have 
been less work and more reliable to use UID and PID instead of 
time.  Still not perfect, but at least the user can clean out old
files to prevent collisions. 

I'm planning to put two restrictions.  First, a screen dump must 
go to a new file.  This is harsh, but I think I need it to deal 
with the symlink race problem.  Second, the pathname of the file 
is severely restricted.  As long as the script/host has just a 
little control, it can do nice things for the user.  It doesn't 
need full control of the pathname. 

The pathname will be [dir]/screen[.UID.PID].txt where [dir] is 
required and must be either "/tmp" or "/home/me".  Home would be 
determined from /etc/passwd, not from $HOME.  I'd like to make 
".UID.PID" replaceable within a narrow range.  With these 
restrictions, an external script which started the xterm (see 
/bin/expect) can have just enough control to detect some trigger 
and grab a screen image.  A script should not be forced to do 
extra work between keystrokes to clear the file if a name change 
will be sufficient.
 
Requiring that a new file be created for each screen dump partly 
protects existing files from modification, but doesn't fully deal
with the symlink race condition.  I expect that can be solved 
using the low-level I/O library, (i.e. open instead of fopen) but 
I haven't set up the experiment yet. 

Now, I *don't* think like a cracker, so if I've missed anything, 
please point it out to me!

> not a '?' sequence) or via an APC.

Hmm.  I was thinking...

OSC Ps ; P1 ; P2 ; Pt ST 
where Ps is the unique value to select this service, 
p1 is 0 for "use home", 1 for "use /tmp"
p2 is 0 for mode 0666, 1 for mode 0600
Pt is: empty to use UID.PID
       " " (blank) to eliminate UID.PID
       or 1-8 digits, letters and period.

A malicious host or a broken program could fill a partition with 
files named screen.1.txt, screen.2.txt and so on.  But I think 
that's all it can do.  Is filling the disk a serious enough 
threat to limit the Pt to one character?  That wouldn't be harsh, 
but I think it's unnecessary. 

Why is this a bad idea?

---

From: Paul Williams <[email-masked]>
Date: Tue, 07 Mar 2000 08:41:52 +0000

Curly++ wrote:
> > going to be done it should be performed via a private sequence
> > (preferably not a '?' sequence) or via an APC.
> 
> Hmm.  I was thinking...
> 
> OSC Ps ; P1 ; P2 ; Pt ST

It's a shame that the writers of xterm didn't structure the code space
for OSC in the same way that DEC did with DCS, ie. use the same

  Param ;...; Param Intermediate Final

structure as for CSI, with extra, textual parameters appearing after
Final and before ST. It would've made the parser a lot simpler, because
you'd know unambiguously where the text starts, even with a variable
number of parameters. At the moment, it seems that the existing codes
are of the form

  OSC Ps ; Pt ST

which is OK until you add more Ps (or Pn) parameters before Pt. Then you
have to start doing something special in the parser based on the value
of the first Ps.

(Of course, this decision is historical -- no reflection on the current
maintainers. Never let history get in the way of a good whinge, I say!)

---

From: [email-masked] (Jeffrey Altman)
Date: 7 Mar 2000 14:23:45 GMT

In article <[email-masked]>,
Paul Williams  <[email-masked]> wrote:
: > OSC Ps ; P1 ; P2 ; Pt ST
: > where Ps is the unique value to select this service,
: > p1 is 0 for "use home", 1 for "use /tmp"
: > p2 is 0 for mode 0666, 1 for mode 0600
: > Pt is: empty to use UID.PID
: >        " " (blank) to eliminate UID.PID
: >        or 1-8 digits, letters and period.
: 
: It's a shame that the writers of xterm didn't structure the code space
: for OSC in the same way that DEC did with DCS, ie. use the same
: 
:   Param ;...; Param Intermediate Final
: 
: structure as for CSI, with extra, textual parameters appearing after
: Final and before ST. It would've made the parser a lot simpler, because
: you'd know unambiguously where the text starts, even with a variable
: number of parameters. At the moment, it seems that the existing codes
: are of the form
: 
:   OSC Ps ; Pt ST
: 
: which is OK until you add more Ps (or Pn) parameters before Pt. Then you
: have to start doing something special in the parser based on the value
: of the first Ps.

The whole purpose of the control sequences OSC, APC, DCS, PM, PU1, PU2
are so that something special will be done with the contents.  If you
can represent the command using X3.64 notation then you should not be
using an OSC but should instead be using a private CSI sequence.

Just to set the record straight, the OSC sequences for Set Window Title
and Set Icon Name were developed by DEC for the DECterm and were later 
supported on windowing versions of the VT terminals.

---

From: Paul Williams <[email-masked]>
Date: Tue, 07 Mar 2000 15:27:28 +0000

> using an OSC but should instead be using a private CSI sequence.

You can't represent a command containing a sixel string or filename as a
CSI sequence, so yes, you must use one of the extension mechanisms you
listed above. However, that doesn't stop you structuring these in the
same way as CSI _up to a point_. It makes sense to use as much of the
existing parser for your new sequences, and then just differ at the
point where you need parameters that don't fit the CSI structure. Take a
look at all the DCS sequences defined by DEC, and you'll see that that's
exactly what they did.

> supported on windowing versions of the VT terminals.

Then I'd say that DEC only ever intended their OSCs to have one
selective parameter. Any more than that, and your sequence starts
getting ambiguous, because ';' doesn't have any standard meaning in OSC.

Hmm...are you saying that DEC defined the sequence "OSC Ps ; title BEL"?
Because that's the only version of Set Window Title and Set Icon Name
that xterm used to accept. Or was that miscopied across from the DECterm
implementation?

Which "windowing versions of VT terminals" supported these sequences?

---

From: [email-masked] (Ben Harris)
Date: 7 Mar 2000 16:14:47 GMT

In article <[email-masked]>,
>that xterm used to accept. Or was that miscopied across from the DECterm

My rather vague understanding, from reading the PuTTY sources (PuTTY 
supports both DECterm and xterm title sequences) is that DECterm uses
"OSC Ps L title ST" or somesuch.

-- 
Ben Harris
Unix Support, University of Cambridge Computing Service.
  If I wanted to speak for the University, I'd be in ucam.comp-serv.announce.

---

From: [email-masked] (Jeffrey Altman)
Date: 7 Mar 2000 16:27:46 GMT

In article <[email-masked]>,
Paul Williams  <[email-masked]> wrote:
: 
: You can't represent a command containing a sixel string or filename as a
: CSI sequence, so yes, you must use one of the extension mechanisms you
: listed above. However, that doesn't stop you structuring these in the
: same way as CSI _up to a point_. It makes sense to use as much of the
: existing parser for your new sequences, and then just differ at the
: point where you need parameters that don't fit the CSI structure. Take a
: look at all the DCS sequences defined by DEC, and you'll see that that's
: exactly what they did.

And when you can do that you use a DCS instead of an OSC or APC.  
DEC was very consistent about this.  All DCS sequences follow X3.64 and
the others do not.

: > Just to set the record straight, the OSC sequences for Set Window Title
: > and Set Icon Name were developed by DEC for the DECterm and were later
: > supported on windowing versions of the VT terminals.
: 
: Then I'd say that DEC only ever intended their OSCs to have one
: selective parameter. Any more than that, and your sequence starts
: getting ambiguous, because ';' doesn't have any standard meaning in OSC.
: 
: Hmm...are you saying that DEC defined the sequence "OSC Ps ; title BEL"?
: Because that's the only version of Set Window Title and Set Icon Name
: that xterm used to accept. Or was that miscopied across from the DECterm
: implementation?

DEC defined it.
 
  DECSWT (Set Window Title)
  DECSIN (Set Icon Name)

: Which "windowing versions of VT terminals" supported these sequences?

VT5xx has it.  And I think the VT420 had it but I don't have a manual to 
check against at the moment.

---

From: Paul Williams <[email-masked]>
Date: Tue, 07 Mar 2000 16:32:49 +0000

> DEC was very consistent about this.  All DCS sequences follow X3.64 and

But this was _only_ a DEC decision. IIRC, DCS is not defined by X3.64 to
have any more structure than OSC or APC.

---

From: "T.E.Dickey" <[email-masked]>
Date: Tue, 07 Mar 2000 20:34:01 GMT

> posted to bugtraq, but the method seems broken to me.  Also, I 

briefly, the point of the patch is to ensure that the effective user owns
the file - and no one else.  if the file is in a directory that prevents
non-owners from deleting the file, then that covers most of the remaining
issues.  (I added a bit to that though - it's in the most recent patch
I made for xterm).

---

From: "T.E.Dickey" <[email-masked]>
Date: Tue, 07 Mar 2000 20:42:40 GMT

> "OSC Ps L title ST" or somesuch.

yes (OSC is supposed to end with ST, DECterm does the Right Thing).  I
had the impression that Sun's use of this sort of escape sequence
preceded DEC's - but since both were implemented ~12 years ago, I'm
not sure (I was using Apollo's at the time, and didn't much like X or
xterm ;-).

-- XFree86 xterm recognizes both...