http://invisible-island.net/xterm/
Copyright © 2022 by Thomas E. Dickey
Here is the latest version of this file.
Ilya Zakharevich and Joe Allen started this feature, on different occasions. I made suggestions, helping them revise their patches, and made my own changes as I adapted their patches.
Joe Allen's contribution is more well-known, though not the beginning of the story. Here is the initial mail which I received from Joe Allen:
Date: Fri, 25 Mar 2005 11:53:37 -0500 (EST) From: Joseph H Allen <jhallen@theworld.com> To: dickey@invisible-island.net Subject: cut & paste enhancements for JOE Hi, I'm the author of JOE (Joe's Own Editor). Do you still maintain xterm? I'm thinking of modifying xterm to better support my editor, and I'm wondering how you feel about including any new changes (or perhaps, I don't need them and you can tell me about features I've missed which cover these). First change: cut & paste bracketing. I'd like a mode which, when set, brackets each paste with a control sequence, so that paste input can be distinguished from normal keyboard input. Something like: ESC [ P ... paste data ... ESC [ Q With this, JOE could temporarily turn off autoindent mode, so that pasted source code is not doubly-indented. VIM has this problem as well. Also in JOE: ` is a prefix key (control key quote), but this screws up paste. With the bracketing, I can temporarily override this meaning of `. Second change: I've added xterm mouse support to JOE, which is nice, but I still want middle-click to be paste. Perhaps I should be able to send a sequence to xterm which causes it to send the paste data- so I receive the middle-click sequence and then send the give-me-paste-data sequence back. Also, with mouse support enabled, I can select blocks of text within JOE (left-click drag), but these are not X clipboard selections, so they can't be pasted into other applications. So, I should be able to send xterm a control sequence, followed by text, which causes xterm to claim ownership of the current X selection, and use the given text for pastes into other applications. Finally, it would be nice if there was a mode where left-click-drag would generate cell coordinates even if the mouse goes beyond the border of the window. I could use this for the typical keep selecting more text thing, where the distance between the mouse and the border gives the scroll rate. There needs to be some way to give negative coordinates when the mouse goes past the top or left edge. So what do you think? Would accept a patch for these things? (BTW, I'm impressed with how well xterm has been keeping up with internationalization support). Joe
Joe Allen's git repository has the suggested patches, but its history starts in April 2010, and as of February 2022, the most recent commit was in April 2015. The original patches (with some history) can be found on Oskari Saarenmaa's "complete" joe-editor repository (which covers the range from September 1988 to January 2011):
This is rather lengthy, but compare with my updates for xterm in patch #203,
- make paste of UTF-8 faster for Western character sets by checking range of incoming data (patch by Joe Allen).
- add experimental option to allow applications to get or set the selection data as a BASE64 string (adapted from patch by Joe Allen).
Those can be seen in the commit which I labeled as xterm-202f:
The first item was a small change to the code which
told xterm to not check for zero-width
combining characters if the code was less than
0x300
.
The second item reflects my suggestion to encode the selection data (to guard against control characters) and to improve the delimiters sent (to make them less likely to be part of the data which is returned to the application).
xterm-readme describes Joe Allen's proposed patch.
I modified some parts and omitted some of this:
xterm-patch should be applied to xterm-200, which can be found at: http://dickey.his.com/xterm/xterm.html With this patch, the following escape sequences become available, and you can enable "-joexterm" in the joerc file: ESC [ ? 2 0 0 4 h Enable bracketed pastes. ESC [ ? 2 0 0 4 l Disable bracketed pastes. When bracketed pastes are enabled, paste data looks like this: ESC [ 2 0 0 ~ <paste-text> ESC [ 2 0 1 ~ ESC [ ? P Send base64 encoded paste data to program (when JOE gets middle button click, it sends this to xterm to make it send the paste data). ESC [ ? 2 P Accept base64 encoded paste data until ESC. This data becomes the "selected text" which can be pasted by other applications. ESC [ ? 2 0 0 7 h Enable outside of frame mouse coordinates. ESC [ ? 2 0 0 7 l Disable outside of frame mouse coordinates. When outside of frame mouse coordinates are enabled, and xterm mouse support is enabled (ESC [ ? 1 0 0 2 h), mouse coordinates are sent even if they are outside of xterm's window frame. The coordinate code bytes should be interpreted like this (we need negative coordinates in case mouse is above or to left of window): Code Coordinate ---- ---------- 33 - 240 1 - 208 (normal) 32 0 (left/top of frame) 241 - 255 -15 - -1 (left/top of frame) (When this mode is off, codes 33 - 255 correspond to coordinates 1 - 223). Also, this patch makes XTerms compiled with wide character support much faster. Also, this patch makes XTerm stores UTF-8 data in the server's cut buffer, instead of latin1.
I modified the escape sequences for setting/getting the
“paste data” by making that part of OSC
(operating system commands), i.e.,
OSC 52
(manipulate selection data). Joe
Allen's patch built on, but did not mesh with an existing feature
for readline added a few years before (i.e., the references to
SCREEN_FLAG(screen,paste_brackets)
), in patch #167 (I repaired this, hence
“adapted”).
I had made this a non-default compile-time feature because I was
not satisfied with the documentation. From our email:
Date: Tue, 1 Oct 2002 18:23:55 -0700 From: Ilya Zakharevich <ilya@Math.Berkeley.EDU> To: Thomas Dickey <dickey@herndon4.his.com> Subject: Re: HOWTO: Mouse editing with readline in xterm On Tue, Oct 01, 2002 at 07:10:48PM -0400, Thomas Dickey wrote: > On Tue, Oct 01, 2002 at 03:43:50PM -0700, Ilya Zakharevich wrote: > > Here is how to enable mouse-edition of the command line in recent xterm's: > > > > a) After running ./configure, add -DOPT_READLINE to the Makefile. [It > > is a bug in configure that one cannot do this easier ;-] > > > > b) This enables the following new [undocumented :-(] control codes for > > DECSET etc: > > yes, I'd like to see documentation (and some sort of test application > smaller than emacs ;-) > > > Here is the example for tcsh: > > but what does it _do_? This is described in the part of the message you conveniently deleted. ;-) 2001 on: click-1 "moves point" 2002 on: click-2 "moves point" before doing paste 2003 on: double-click-3 "moves point" to the end of selection, then emits enough BackSpace characters to erase the selection 2004 on: Emits escape sequences for F200/F201 before/after doing paste 2005 on: When pasting, emit quote-char before any pasted char. 2006 on: When pasting, emit "\n" as C-j (usually emits C-m) All the mouse bindings are active on the same line as the insertion cursor only (including wrapped lines). "move point" means "emits enough Left/Right keysequences to move the point in ReadLine applications". Ilya
The line with “2004” is the bracketed paste feature. The rest is a (too) terse summary of the features usable from readline. A variation of that was in the source-code:
#define SET_BUTTON1_MOVE_POINT 2001 /* click1 emit Esc seq to move point*/ #define SET_BUTTON2_MOVE_POINT 2002 /* press2 emit Esc seq to move point*/ #define SET_DBUTTON3_DELETE 2003 /* Double click-3 deletes */ #define SET_PASTE_IN_BRACKET 2004 /* Surround paste by escapes */ #define SET_PASTE_QUOTE 2005 /* Quote each char during paste */ #define SET_PASTE_LITERAL_NL 2006 /* Paste "\n" as C-j */
None of that went into ctlseqs.ms
. Joe Allen's
update to ctlseqs.ms
established the name as
bracketed paste.
I omitted the feature for using UTF-8 in a cut-buffer. The ICCCM is fairly explicit about the encoding, saying in Chapter 3. Peer-to-Peer Communication by Means of Cut Buffers
The cut buffers consist of eight properties on the root of screen zero, named by the predefined atoms CUT_BUFFER0 to CUT_BUFFER7. These properties must, at present, have type STRING and format 8.
and in Text Properties:
STRING as a type or a target specifies the ISO Latin-1 character set plus the control characters TAB (octal 11) and NEWLINE (octal 12). The spacing interpretation of TAB is context dependent. Other ASCII control characters are explicitly not included in STRING at the present time.
I omitted the feature for out-of-frame mouse events, because it did not address how to make this work when xterm lost focus, as well as because it reduced the available range for mouse coordinates:
I addressed the latter issue with the SGR-mouse 1006 mode in patch #277, increasing the range of coordinates.
Presumably Joe Allen kept the readme and patch in the sources, in case the issue of focus could be resolved. But because he did not update and revise the patch, I did not reconsider that part of the changes.
A couple of years later, I added control sequences which could be used to notify an application that the mouse pointer had moved in or out of the window (patch #224):
- add control sequences for enabling/disabling focus in/out event reporting (request by Bram Moolenaar).
Here is a crude timeline showing programs which adopted bracketed paste (with the caveat that these are commit-dates, not release-dates).
2002-08-24 xterm #167 (introduced) 2005-04-06 joe 2005-06-21 xterm #203 (documented) 2008-01-20 urxvt 2009-12-28 vte 2010-04-24 mintty 2010-05-09 teraterm 2011-07-20 nsterm 2011-07-24 mlterm 2011-09-03 iTerm2 2012-02-19 putty 2012-03-03 tmux 2013-03-15 screen 2013-04-07 vim-bracketed-paste 2013-05-29 mosh 2013-09-23 st 2014-01-06 terminology 2014-01-11 konsole 2014-04-17 emacs 2014-07-21 hterm 2014-11-03 bash 2015-02-17 neovim 2015-06-19 zsh 2015-12-16 DomTerm 2016-11-12 kitty 2016-11-28 alacritty 2017-01-21 vim 2017-10-29 xterm.js 2018-04-22 terminator 2019-06-22 foot 2021-02-08 ms-terminal
There are several categories of programs: editors, terminals, quasi-terminals and shells. The lists for each category are sorted alphabetically.
Joe Allen's text editor was his initial motivation for the feature. Other editors have the same issue. Quoting from The VI User's Handbook (1984) description of the autoindent mode:
If ai ... in Input Mode, each new line will start in the same column as the 1st non-white character in the preceding line.
The markers sent by the terminal to delimit pasted text give the editor a clue that it should temporarily turn autoindent off.
commit bfc30790686607fac1b7667d3a73d0f46b80e85f Author: Daniel Colascione <dancol@dancol.org> Date: Thu Apr 17 00:54:23 2014 -0700 2014-04-17 Daniel Colascione <dancol@dancol.org> Add support for bracketed paste mode; add infrastructure for managing terminal mode enabling and disabling automatically. * xt-mouse.el: (xterm-mouse-mode): Simplify. (xterm-mouse-tracking-enable-sequence) (xterm-mouse-tracking-disable-sequence): New constants. (turn-on-xterm-mouse-tracking-on-terminal) (turn-off-xterm-mouse-tracking-on-terminal): Use tty-mode-set-strings and tty-mode-reset-strings terminal parameters instead of random hooks. (turn-on-xterm-mouse-tracking) (turn-off-xterm-mouse-tracking): Delete. * term/xterm.el (xterm-extra-capabilities): Fix bitrotted comment. (xterm-paste-ending-sequence): New constant. (xterm-paste): New command used for bracketed paste support. (xterm-modify-other-keys-terminal-list): Delete obsolete variable. (terminal-init-xterm-bracketed-paste-mode): New function. (terminal-init-xterm): Call it. (terminal-init-xterm-modify-other-keys): Use tty-mode-set-strings and tty-mode-reset-strings instead of random hooks. (xterm-turn-on-modify-other-keys) (xterm-turn-off-modify-other-keys) (xterm-remove-modify-other-keys): Delete obsolete functions. * term/screen.el: Rewrite to just use the xterm code. Add copyright notice. Mention tmux.
commit dbd5b60cbeea9f1d65dc1d14c4ab427725f7aab6 Author: Joseph Hallen <jhallen@users.sourceforge.net> Date: Wed Apr 6 16:37:29 2005 +0000 new base64 encoded xterm-patch
however, removing the use of bracketed paste (while retaining the out-of-frame feature):
commit a78c10fac9a522eefab5493741a169de9e4ffd49 Author: Joseph Hallen <jhallen@users.sourceforge.net> Date: Wed Apr 6 16:40:31 2005 +0000 one more thing diff --git a/mouse.c b/mouse.c index cbebcb5..ccf3c8e 100644 --- a/mouse.c +++ b/mouse.c @@ -689,7 +689,7 @@ void mouseopen() if (usexmouse) { ttputs(US "\33[?1002h"); if (joexterm) - ttputs(US "\33[?2004h\33[?2007h"); + ttputs(US "\33[?2007h"); ttflsh(); } #endif @@ -700,7 +700,7 @@ void mouseclose() #ifdef MOUSE_XTERM if (usexmouse) { if (joexterm) - ttputs(US "\33[?2007l\33[?2004l"); + ttputs(US "\33[?2007l"); ttputs(US "\33[?1002l"); ttflsh(); }
commit 59fb8f81723f88935c3b4c7a1c0df4d6db1c2ff8 Author: Thiago de Arruda <tpadilha84@gmail.com> Date: Tue Feb 17 23:07:11 2015 -0300 tui: Add support bracketed paste Inspired by the vim-bracketed-paste plugin but adapted for the new TUI. Also initialize some variables of type `Error` that were uninitialized
The repository for the vim-bracketed-paste plugin was created in 2013:
commit 46bb985dc784d95481cdf4211f97564735df99f2 Author: Conrad Irwin <conrad.irwin@gmail.com> Date: Sun Apr 7 16:56:00 2013 -0700 vim-bracketed-paste!
The script used in the plugin came from an example by another developer in Debian bug report #504244 vim: Please support turning on and using xterm bracketed paste mode. That report began in November 2008, and was completed in August 2017. The plugin has a comment which points to additional sources of information:
" Code from: " http://stackoverflow.com/questions/5585129/pasting-code-into-terminal-window-into-vim-on-mac-os-x " then https://coderwall.com/p/if9mda " and then https://github.com/aaronjensen/vimfiles/blob/59a7019b1f2d08c70c28a41ef4e2612470ea0549/plugin/terminaltweaks.vim " to fix the escape time problem with insert mode. " " Docs on bracketed paste mode: " http://www.xfree86.org/current/ctlseqs.html " Docs on mapping fast escape codes in vim " http://vim.wikia.com/wiki/Mapping_fast_keycodes_in_terminal_Vim
The comments in the plugin credit a Stack Overflow answer by Chris Page (August 13, 2011), which does not mention Josh Triplett's 2008 bug report that provided the snippet.
commit ec2da36ca48b40c0654b32a8d2c9f52e796daa5e Author: Bram Moolenaar <Bram@vim.org> Date: Sat Jan 21 20:04:22 2017 +0100 patch 8.0.0210: no support for bracketed paste Problem: Vim does not support bracketed paste, as implemented by xterm and other terminals. Solution: Add t_BE, t_BD, t_PS and t_PE
There was some discussion in 2014 in vim-dev about providing support directly in vim. Some of the information is incorrect:
Vim documents the added variables in the help-file section xterm-bracketed-paste.
Most of the terminals listed here are designed to imitate
xterm, and abuse that relationship by setting the TERM
environment variable to imply a better match than actually
exists. In this list, I refer to each by the name used in
ncurses. However:
Not all of these are “X terminals” — some are designed for other systems: iTerm2, nsterm, ms-terminal, and putty.
There are no terminfo/termcap capabilities for the
feature. Vim's variables assume that a termcap description
might provide different values (omitting the
“t_
” prefix), but other editors
simply use hard-coded strings.
commit ff5081d5e5cd2ddcf775357853b81afbcf45aef4 Author: Joe Wilm <joe@jwilm.com> Date: Mon Nov 28 14:39:37 2016 -0800 Add support for bracketed paste Binding/Action execute now has access to TermMode to support bracketed paste mode.
commit 420ff7a335ba2869d1fd06879e8e11dc2b17f43a Author: “Per <“per@bothner.com”> Date: Wed Dec 16 11:07:22 2015 -0800 Re-implement alternate buffer mode. Improve special-key handling.
commit 6aba78a145218f30cfdf0c9d58799abbd3a832c4 Author: Daniel Eklöf <daniel@ekloef.se> Date: Sat Jun 22 22:25:50 2019 +0200 csi: wip: recognize the private CSIs for bracketed paste, smcup/rmxup, smkx/rmkx
and
commit 482c8f76a99d85970a40f41e455fc6be80b2a5ba Author: Daniel Eklöf <daniel@ekloef.se> Date: Thu Jul 11 12:33:31 2019 +0200 selection: add support for pasting from clipboard
commit a063b208982e96624eb011da7124cf7e3e1f8a05 Author: Robert Ginda <rginda@chromium.org> Date: Mon Jul 21 11:08:25 2014 -0700 hterm: Implement bracketed paste mode. * DECSET/DECRST mode 2004 toggles bracketed paste mode. When enabled, pastes are sent as "ESC [ 2 0 0 ~ <paste-data> ESC [ 2 0 1 ~". * Add test. Change-Id: Ibacf433c4bc3fe71566b78f26b374224d6704c32 Reviewed-on: https://chromium-review.googlesource.com/209156 Reviewed-by: Marvelous Marius <mschilder@chromium.org> Reviewed-by: Robert Ginda <rginda@chromium.org> Tested-by: Robert Ginda <rginda@chromium.org>
commit 72950c7800122ae896cbeb5cd2e923774c89e8ef Author: George Nachman <git.xyz@georgester.com> Date: Sat Sep 3 22:33:09 2011 -0700 Implement bracketed paste mode diff --git a/Headers/iTerm/VT100Terminal.h b/Headers/iTerm/VT100Terminal.h index 7933d86b1..2c84a9139 100644 --- a/Headers/iTerm/VT100Terminal.h +++ b/Headers/iTerm/VT100Terminal.h @@ -329,6 +329,9 @@ typedef enum { //terminfo char *key_strings[TERMINFO_KEYS]; + + // http://www.xfree86.org/current/ctlseqs.html#Bracketed%20Paste%20Mode + BOOL bracketedPasteMode_; } + (void)initialize;
The url in the source-code caught my eye, because my control sequences web-page was linked to the FAQ since March 2007. I stopped updating XFree86 in August 2006 (patch #216).
As you may see on the Internet Archive, besides adding new
features, I spent time improving the means for transforming
ctlseqs.ms
(an nroff file) to a useful webpage. Some of that is outlined in
my page on man2html. For the
rest, well that is done by a half-dozen Perl scripts.
Failing to find the documentation for a program that you're copying, for more than four years does not provide much confidence in the results. Others mentioned on this page have the same issue.
Its developer began implementing bracketed paste in November 2016
commit 2c0523246d06af85641c80c86518430768d01582 Author: Kovid Goyal <kovid@kovidgoyal.net> Date: Sat Nov 12 13:52:18 2016 +0530 More work on making Screen native
citing Conrad Irwin's blog for that feature (though citing xterm for others):
// Bracketed paste mode // http://cirw.in/blog/bracketed-paste #define BRACKETED_PASTE (2004 << 5) #define BRACKETED_PASTE_START "\033[200~" #define BRACKETED_PASTE_END "\033[201~"
Another developer revised it in June 2018:
commit 668f6fa257d638913d8c5b3e72f2c3c8a15498b4 Author: Daniel Colascione <dancol@dancol.org> Date: Fri Jun 8 04:52:40 2018 -0700 More robustly strip bracketed paste termination sequence The previous code performed only one replacement on the bracketed paste content. This procedure didn't stop someone embedding the terminating sequence anyway. POC: 1) $ x=$'\033[201~'; printf '%s%s%s\necho hello world\n' "${x:0:1}" "$x" "${x:1}" | xclip 2) paste into kitty 3) see the shell execute a command! This patch closes this hole.
commit 8082bcd2ef822fbbe06ca8716633f44fd20c0cb4 Author: Kurt Hindenburg <kurt.hindenburg@gmail.com> Date: Sat Jan 11 12:28:46 2014 -0500 Add xterm's bracketed paste mode Allow xterm's bracketed paste mode to work When enabled by '\e[?2004h', pasted data is surrounded by \e[200~ and \e[201~. This is mainly for text editors to temporarily switch off autoindent and line wrapping. http://invisible-island.net/xterm/ctlseqs/ctlseqs.html This page have a .vimrc that will trigger this https://bugzilla.gnome.org/show_bug.cgi?id=605299 Thanks to Egmont Koblinger for patch BUG: 324946 FEATURE: 4.13
commit 2a32837cca0250ca474d33ffd0247b024960e0ff Author: andy.koppe <andy.koppe@2782a51e-c42b-11dd-927d-974589228060> Date: Sat Apr 24 05:32:41 2010 +0000 Add xterm bracketed paste mode: http://invisible-island.net/xterm/ctlseqs/ctlseqs.html#Bracketed%20Paste%20Mode
commit 95fa9debef9860a21d5a255970f127c93cfd1a54 Author: arakiken <devnull@localhost> Date: Sun Jul 24 18:36:41 2011 +0900 * ml_vt100_parser.c, ml_term.[ch], x_screen.c: Bracketed paste mode (CSI ? 2004 h , CSI ? 2004 l) is supported. * version.h.in: 3.0.6 -> 3.0.6 post
commit 124cbd9e4758a22abe12fb9947d4032db7b3146d Author: Chester Liu <skyline75489@outlook.com> Date: Fri Jan 22 13:11:11 2021 +0800 Add skeleton code for bracketed paste mode (#8840) This adds the skeleton code for "bracketed paste mode" to the Windows Terminal. No actual functionality is implemented yet, just the wiring for handling DECSET/DECRST 2004. References #395 Supersedes #750
and
commit 2c603ef953752b315f3c1e1c4b49368c76390cf3 Author: Chester Liu <skyline75489@outlook.com> Date: Mon Feb 8 21:11:01 2021 +0800 Add support for paste filtering and bracketed paste mode (#9034) This adds "paste filtering" & "bracketed paste mode" to the Windows Terminal. I've moved the paste handling code in `TerminalControl` to `Microsoft::Console::Util` to be able to easily test it, and the paste transformer from `TerminalControl` to `TerminalCore`. Supersedes #7508 References #395 (overall bracketed paste support request) Tests added. Manually tested.
According to Chris Page's Stack Overflow answer to Pasting code into terminal window into vim on Mac OS X in August 2011
As of Mac OS X Lion 10.7, Terminal supports “bracketed paste mode,” which enables the terminal emulator to tell the program connected to the tty when the user pastes text, so that the program won't interpret it as editing commands. Programs that support it send the terminal an escape sequence to enable this mode, in which the terminal surrounds pasted text with a pair of escape sequences that identify the start and end.
That version of MacOS had been released a few weeks earlier (July 20, 2011).
commit 21b04f5e00e21b4ffbb790d23c26e25b7f977ba2 Author: Simon Tatham <anakin@pobox.com> Date: Sun Feb 19 10:27:18 2012 +0000 Patch from Matsui Nag to implement xterm's "bracketed paste mode", in which text pasted into the terminal is preceded and followed by special function-key-like escape sequences ESC[200~ and ESC[201~ so that the application can identify it and treat it specially (e.g. disabling auto-indent-same-as-previous-line in text editors). Enabled and disabled by ESC[?2004h and ESC[?2004l, and of course off by default. [originally from svn r9412]
commit 8ac0a5f872b0024bab161fa020126013724758ba Author: Egmont Koblinger <egmont@gmail.com> Date: Mon Sep 23 09:22:47 2013 +0200 Add bracketed paste mode This patch enables bracketed paste mode ( http://invisible-island.net/xterm/ctlseqs/ctlseqs.html#Bracketed%20Paste%20Mode ). It's mainly useful for text editors to disable line wrapping and auto indentation when text is being pasted, rather than typed from keyboard. On the emulator side, it is supported by at least xterm, urxvt, gnome-terminal, putty, iterm2; and I have a patch for konsole. On the application side, vim can be configured easily to handle this, and I have pending patches for mcedit and joe. Probably many others also support it.
The comment about joe is odd, but looking at its history, it seems that Joe Allen removed its use of bracketed paste in April 2005. Egmont restored it nine years later:
commit 036fb8cbbf4f03e6f8776d3dc027c39525bbdcda Author: John J. Jordan <jjjordan@users.sf.net> Date: Mon Nov 17 00:50:54 2014 -0800 Support bracketed paste (patch by Egmont Koblinger).
commit ac4117ce8a2ef388a067c7d9882438edcfad60e9 Author: Boris Faure <billiob@gmail.com> Date: Mon Jan 6 21:51:23 2014 +0100 add bracketed paste (by egmont). Closes T670
commit b8970cfd5d5e93cc809a8b143721e08c5d57f85b Author: Emanuele Giaquinta <emanuele.giaquinta@gmail.com> Date: Sun Jan 20 14:12:39 2008 +0000 implement bracketed paste mode (xterm private mode 2004).
commit 4db015d59e80a992c86827e612e06c7df1c901f0 Author: Josh Triplett <josh@joshtriplett.org> Date: Mon Dec 28 03:22:48 2009 +0100 Bug 605299 - Please support xterm bracketed paste mode
commit 2800513919ff9927b5dd8bc38a303a3bafc9617c Author: doda <doda@f5f01b69-1e22-0410-acbf-894ab4bd6246> Date: Sun May 9 09:21:21 2010 +0000 Bracketed Paste Mode をサポート。 対応しているホスト側のアプリケーションで、キー入力と貼り付けの区別が付けられるようになる。 .vimrcの例: if &term == "xterm" let &t_ti = &t_ti . "\e[?2004h" let &t_te = "\e[?2004l" . &t_te set pastetoggle=<Esc>[201~ function XTermPasteBegin(ret) set paste return a:ret endfunction map <special> <expr> <Esc>[200~ XTermPasteBegin("i") imap <special> <expr> <Esc>[200~ XTermPasteBegin("") endif 参考: https://bugzilla.gnome.org/show_bug.cgi?id=605299
commit f9bd301f96e2a5a9ffc32eb40e28ed2167f13402 Author: Martin Dorey <martindorey@gmail.com> Date: Sun Apr 22 10:16:44 2018 -0700 terminator/src/terminator/model/TerminalModel.java: terminator/src/terminator/terminal/escape/CSIEscapeAction.java: terminator/src/terminator/view/TerminalView.java: Markus Laker points out https://lwn.net/Articles/749992/, which mentions a "bracketed paste mode" that's news to me. I have a tinfoil hat to protect myself from malicious websites, but many's the time I've suffered a pasting accident, thanks to my human frailties or the 1% of time that things go wrong with the clipboard synchronization code in Citrix Receiver or VNC. Here, then, I add minimal support, following https://cirw.in/blog/bracketed-paste. There's no option or anything to disable the support but a terminal application has to ask for it, which involves adding: set enable-bracketed-paste on ... to ~/.inputrc for Bash 4.4+, which means Debian Stretch but not Jessie, where the .inputrc line is silently ignored. When it works, the behavior is that the lines you paste aren't executed until you press Enter, like you did Ctrl-V Ctrl-J instead of each newline. To get automated :set paste in Vim requires jumping through all these hoops, if not more (I lost track!): sudo aptitude install vim-pathogen mkdir -p ~/.vim/bundle cd ~/.vim/bundle git clone https://github.com/ConradIrwin/vim-bracketed-paste ... and creating a ~/.vimrc containing: execute pathogen#infect() syntax on filetype plugin indent on The first line turns on the new thing. The second line reenables the syntax coloring that you get by default unless you have a ~/.vimrc. I didn't care enough about the third line to investigate. Now I presume that my pastes into remote systems will all include visible escape sequences that will hack me off until I provide some UI to turn the feature off for the current terminal. At least it shouldn't affect anyone who doesn't explicitly edit .inputrc or .vimrc or whatever.
commit 1dbcf70cee9ae88c69cf9724745cbdb7e5364dcc Author: Dan Brown <jdanbrown@gmail.com> Date: Sun Oct 29 23:29:39 2017 -0700 Support bracketed paste mode - As described in https://cirw.in/blog/bracketed-paste - Example support in vim: - https://github.com/ConradIrwin/vim-bracketed-paste/blob/master/plugin/bracketed-paste.vim - http://vimdoc.sourceforge.net/htmldoc/term.html#raw-terminal-mode - Required for e.g. ipython, via python-prompt-toolkit: - https://github.com/ipython/ipython - https://github.com/jonathanslenders/python-prompt-toolkit Repro: - Run `ipython` (version 6.2.1) - Copy multiple lines of text and paste into ipython prompt, e.g. ```py print(1) print(2) print(3) ``` - Before: first line would paste and run, and subsequent lines are lost - After: all lines paste into single prompt
These are applications which run in a terminal to provide additional features (multiplexing connnections or maintaining a persistent connection). Because they attempt to provide applications (e.g., text editors) with a consistent terminal interface, they also transform control sequences used by the application to/from the underlying terminal.
commit c6bf3a2025e86b34512a995ea8b1e45d7586860f Author: Barosl LEE <vcs@barosl.com> Date: Wed May 29 12:54:29 2013 +0900 Implement bracketed paste mode Allow bracketed paste mode-setting control sequences to be passed to the outer terminal. Signed-off-by: Barosl LEE <vcs@barosl.com> Closes #430
commit c389013e89e7b04ce43872f2e72d43f77461a3c0 Author: Hayaki Saito <user@zuse.jp> Date: Fri Mar 15 16:23:54 2013 +0100 Support "bracket paste mode" and cursor-style manipulation Hello, lists This patch adds the following two features to GNU Screen: - Bracket Paste Mode (DECSET/DECRST 2004) - DECSCUSR(cursor style manipulation) By using "bracketed paste mode", the pasted text is bracketed with special control sequences. DECSCUSR can change cursor style and shape (blink/steady, block/Vertical bar/horizontal bar). ref: http://invisible-island.net/xterm/ctlseqs/ctlseqs.html These days, many of xterm-compatible terminal emulators support these features. But current GNU Screen blocks them. This patch manages states of "Bracket Paste Mode (DECSET/DECRST 2004)" and DECSCUSR(cursor style manipulation), for each of screens. Please check it. Hayaki Saito <user@zuse.jp> https://lists.gnu.org/archive/html/screen-devel/2013-03/msg00000.html
commit f4fdddc9306886e3ab5257f40003f6db83ac926b Author: Nicholas Marriott <nicm@openbsd.org> Date: Sat Mar 3 09:43:22 2012 +0000 Support "bracketed paste" mode. This adds a -p flag to paste-buffer - if this is used and the application has requested bracketed pastes, then tmux surrounds the pasted text by \033[200~ and \033[201~. Applications like vim can (apparently) use this to avoid, for example, indenting the text. From Ailin Nemui.
The motivation for adding this feature to shells is that they provide (limited) support for editing commands.
commit a0c0a00fc419b7bc08202a79134fcd5bc0427071 Author: Chet Ramey <chet.ramey@case.edu> Date: Thu Sep 15 16:59:08 2016 -0400 Bash-4.4 distribution sources and documentation
seen earlier in the bash-4.4-testing branch:
commit f3aad56dceb53b14c8bb3e73d085ee45e39486d7 Author: Chet Ramey <chet.ramey@case.edu> Date: Mon Nov 3 14:32:12 2014 -0500 commit bash-20141031 snapshot
The change-log mentions this:
lib/readline/bind.c - enable-bracketed-paste: new bindable variable, enables support for a terminal's `bracketed paste mode'. Code contributed by Daniel Colascione <dancol@dancol.org>
commit 98687fa1dec803f041cbb5417c146d8aa5129b53 Author: Oliver Kiddle <opk@zsh.org> Date: Fri Jun 19 00:15:38 2015 +0200 35474, 35492: support the bracketed paste mode of newer terminal emulators
Zsh's safe-paste plugin additionally attempts to protect the user against unexpected behavior while editing.
In his blog bracketed paste mode (April 2013), Conrad Irwin stated
For a long time bracketed paste mode was only supported by xterm and its derivatives (urxvt, etc.), but as of recently (a year or so ago) support has been added to most other terminals, including libvte which powers gnome-terminal et.al., iTerm 2 and Terminal.app. In other words, almost every terminal in widespread use now suppports bracketed paste mode.
Actually no:
That “almost every terminal in widespread use” is hyperbole rather than fact. No one has done a complete survey of the topic. For those inclined to do so, start here.
It is easy to find counter examples in the context of April 2013 (Cygwin console, Linux console, actually any operating system console).
That “a year or so” would only be accurate for putty as one can see from the timeline summarized here.
The blog cited the old (2006) version of the control sequences document. That was seven years out of date.
It goes downhill from there.
But the blog made me curious to find when various programs adopted the feature. In collecting data, I noticed that the script used in the vim plugin was actually written by someone else, but that in turn was not the original script.
Debian bug 504244
report by Josh Triplett provided the original script.
GNOME bug 605299
also by Triplett repeats the script.
Stack Overflow
answer by Chris Page uses a slightly modified version of
Triplett's script (e.g., allowing TERM
to match
things such as xterm-color, and tidying up with
cmap
). Vim plugin used that script.
Here are the two side-by-side:
Triplett | Page/Irwin |
---|---|
if &term == "xterm" let &t_ti = &t_ti . "\e[?2004h" let &t_te = "\e[?2004l" . &t_te function XTermPasteBegin(ret) set pastetoggle=<Esc>[201~ set paste return a:ret endfunction map <expr> <Esc>[200~ XTermPasteBegin("i") imap <expr> <Esc>[200~ XTermPasteBegin("") endif |
if &term =~ "xterm.*" let &t_ti = &t_ti . "\e[?2004h" let &t_te = "\e[?2004l" . &t_te function! XTermPasteBegin(ret) set pastetoggle=<Esc>[201~ set paste return a:ret endfunction map <expr> <Esc>[200~ XTermPasteBegin("i") imap <expr> <Esc>[200~ XTermPasteBegin("") cmap <Esc>[200~ <nop> cmap <Esc>[201~ <nop> endif |
Bracketed paste is nothing more than a clue to an application that input (normally from the keyboard) is coming from a select/paste operation.
It has limitations:
It does not tell the application how long the pasted text is.
It does not ensure that the application can distinguish pasted text from concurrent keyboard input.
The terminating marker might be embedded in the pasted text.
That last part was not a problem initially:
Selections from xterm's window will give only printable characters, along with newlines. There are no escape characters. There are no tab characters, either.
Conceivably, some user could (with a command-line application) load the X clipboard with random text and use xterm's selectToClipboard feature to paste that into a window.
By default, xterm uses the standard X selection mechanism. Other terminals differ, of course, influenced by Netscape which copied the clipboard feature from Internet Explorer. Netscape's descendents (and imitators) are still around of course. The selectToClipboard feature of xterm accommodates those. Other terminals simply use the clipboard.
As a result, once web browsers were introduced as a primary end-user interface, select/paste became an issue. Not understanding the limitations of bracketed paste, they assumed that since a given text-editor was doing a passable job of handling newlines, when bracketed paste was enabled, that it was safe to use in any application. More of these people paste directly into a shell command-line (Stack Overflow for instance is a ready source of one-liners), and assume everything will work just as it shows on the webpage from which they copied the command.
There were a few problems with this approach, for instance
Date: Sun, 14 Apr 2013 16:46:37 -0400 From: Thomas Dickey <dickey@his.com> To: SaitoHayaki <saitoha@me.com> Cc: dickey@his.com Subject: Re: About bracketed paste mode On Sun, Apr 14, 2013 at 11:32:31PM +0900, SaitoHayaki wrote: > Hello > > Do you think about this issue? > http://thejh.net/misc/website-terminal-copy-paste > > I think, the end sequence of "bracket paste mode" (CSI 2 0 1 ~) should be +escaped in some way. > If the proper way for this issue is established, we can write some patches for +terminal applications for persuading them to provide security and "paste aware" +usability. He already commented on the end-sequence, and given the example, it is irrelevant whether bracketed paste mode is used or not. The fix for this is to not paste into a command-interpreter from an unfiltered source. For the given examples, I'd normally paste into a text-editor (vile or some variant of vi), so I could make a script. However, the given example is written to supply an escape character as part of the text. It might be good to add a feature to xterm to prevent control characters other than cr/lf/tab/ff from being pasted into it. That would help more than modifying the bracketed paste control. Or course it would only fix the problem of safely pasting into a text-editor. Select/paste _from_ vi or lynx running in xterm is pretty reliable. Selecting from Firefox or Internet Explorer only leads to confusion. -- Thomas E. Dickey <dickey@invisible-island.net> http://invisible-island.net ftp://invisible-island.net
This was neither the first issue, nor the last. I added the allowPasteControls resource (patch #292). It is impossible to please everyone. Four years later, A look at terminal emulators, part 1 (March 30, 2018) by Antoine Beaupré mentioned Jann Horn's page, and went on to say
Bracketed paste mode is explicitly designed to neutralize this attack. In this mode, terminals wrap pasted text in a pair of special escape sequences to inform the shell of that text's origin. The shell can then ignore special editing characters found in the pasted text. Terminals going all the way back to the venerable xterm have supported this feature, but bracketed paste also needs support from the shell or application running on the terminal.
That is an inference by the author of the article, not stated in Irwin's blog, which mentions its use in a safe-paste plugin for ZLE. That script attempts to capture codes 0 to 255 (which might be a problem with UTF-8), mapping carriage return to newline. When the bracketed paste end-marker is found, it copies all of the captured codes to the command-buffer. How the shell might interpret special characters in the command-buffer is not mentioned either in the script or in the blog. The blog only mentions newlines and unwanted spaces.
That, and other comments by anarcat (Beaupré) got some disagreement from georg.s (Georg Sauthoff) who disputed Beaupré's assertion that the changes for allowPasteControls did not work, and that bracketed paste was for some reason needed.
If someone finds a problem in the implementation, that is a matter for a bug report. I followed up on Georg's comments (including a pointer to yet another Stack Overflow answer), and added another resource in patch #333, disallowedPasteControls to fine-tune the previous setting.
Both allowPasteControls (all-on) and disallowedPasteControls (selective-off) were independent of bracketed paste, but because of undue expectations on the part of end-users, some assumed a dependency.
Other terminals have similar issues. The main difference is that they are better-documented in xterm.
Don't Trust This Title: Abusing Terminal Emulators with ANSI Escape Characters (January 6, 2022), by Eviatar Gerzi gives an example.
Manipulating the selection (OSC 52) has not gotten as much attention as the bracketed paste feature. The table below gives the program names in the same order as the bracketed paste timeline, to make it more apparent that this feature is neither old nor new.
2005-06-21 full xterm 2006-03-01 full joe ---------- none urxvt ---------- none vte 2016-09-14 full mintty 2011-11-01 full teraterm ---------- none nsterm 2011-11-16 part mlterm 2012-08-13 part iTerm2 ---------- none putty 2017-01-22 part tmux ---------- none screen ---------- none vim-bracketed-paste 2017-07-15 part mosh 2017-03-18 part st ---------- none terminology ---------- none konsole 2015-04-13 part emacs 2012-06-20 part hterm ---------- none bash ---------- none neovim ---------- none zsh 2020-09-23 full DomTerm 2018-05-21 full kitty 2018-01-02 part alacritty ---------- none vim ---------- none xterm.js 2021-09-26 full terminator 2019-07-19 full foot 2020-06-30 part ms-terminal
Counting the cases: