http://invisible-island.net/
Copyright © 1996-2022,2024 Thomas E. Dickey
ded (pronounced "dee–eee–dee") allows you to navigate through multiple file lists or a directory tree, viewing or changing file attributes rapidly. In addition to conventional file information, it operates on the file's RCS or SCCS archives (or CVS or SVN, etc), making it useful for source-control as well as system administration. Curses-based, it runs on UNIX systems.
This is a long-term project. I originally began in 1984, enhancing a version of dired, while I was at the ITT Advanced Technology Center. This was a program in C, written by Stuart Cracraft (SRI) and Jay Lepreau (University of Utah).
/* ********************************************************************** * * * dired [][dir][files] Stuart Cracraft (mclure@sri-unix) Sept 1980 * * * * Directory editor. * * edit/delete files in a directory or a file list * * compile with the berkeley termlib archive * * and don't forget to change the 'helpfile' and * * 'dirednam' strings to reflect your own setup. * * * * Note: if you make improvements, I'd like to get them too. * * Stuart Cracraft mclure@sri-unix, * * ucbvax!menlo70!sri-unix!mclure * * so would I: Jay Lepreau lepreau@utah-20, * * decvax!randvax!utah-cs!lepreau * ********************************************************************** */
There is some dispute over whether that dired is based on the emacs mode introduced around the same time (1980) or the reverse (since neither program's documentation at the time credited the other), but it is indisputable that both were inspired by an earlier stand-alone program running on Tenex available in the Stanford AI Lab (SAIL) in 1978.
As modified by Lepreau, dired had crude display management, which allowed a user to enter shell commands and see them at the bottom of the screen—something that neither curses or termcap was designed to do:
/* * Enhanced by J.Lepreau, Univ of Utah, 8-10/81: * --bunch of stuff, including more cmds, 2 window mode, * bufering output, pathname substition in !escapes, * initial sort options, ^G escape from sorts, ^U escape * from !escapes, empty dir check. */
The 4.2BSD system had a games directory containing a mixture of command-line, termcap and curses programs which I explored briefly in off-hours. The only interesting one was snake. That was a termcap program. Although I was curious about the difference between termcap and curses, the manual pages were not helpful. In each case, the manual page was terse, providing only the function names and a few words to summarize their intended use. The dired program helped introduce me to the curses library. There was a tutorial (mostly a code example of a demo program twinkle) which I found using dired in the BSD source tree. The tutorial was only available in the BSD sources. In other systems (such as Apollo), only the manual page was available.
Among other changes, early in 1984, I modified dired to allow the user to change the fraction of the screen which was devoted to the command output. Later (in October 1984, after I was not actively working on dired and had moved to the project where I wrote flist), an associate asked if I was planning to add a directory-tree display. This was about six months before xtree was introduced; he had apparently seen some earlier program with the feature but was unable to provide details.
There are later versions of dired. However I found nothing useful in the later changes, since the most apparent changes were more complicated (and cumbersome) key-bindings for the functions.
Subsequently, I designed and implemented a series of directory editors including flist, and this program (which uses a directory-tree module, written in 1987, from an earlier program begun at the end of 1985). By then I had seen xtree (but not used it—the designs are different). The current program is not the first ded:
Although dired used BSD
termcap, a newer program might have used curses.
flist uses VMS's SMG$ calls,
and there was no curses library available on VMS for several
years. After writing flist, I began a new
directory editor in late 1985 on a
Sperry PC (probably
using 4.2BSD).
That computer's curses library did not work (it dumped core); so I wrote an imitation of the curses interface using termcap. The wrapper (like the one which I wrote for tin in the mid-1990s) provided an interface using curses's function names, but like other low-level interfaces (such as conio and s-lang) did not provide optimization.
That worked well enough for a month or so before I began development using Unix, e.g., SVr2 systems (NCR Tower, Plexus, Altos) and some which were not SVr2 (Charles River, Xenix). We had ten machines (two NCR Towers, two Plexus – a P/35 and a P/60 and one each of the rest). I do not recall the other three, but had ded working on all machines. The curses library on those machines did work, and I discarded the wrapper.
I began writing the directory-tree module around September 1987, before moving on to the Software Productivity Consortium where I continued developing ded.
When I moved, I kept only the directory-tree because I had begun a newer version using the ring feature mentioned below before obtaining permission from old/new companies to use the older ded.
The term “directory editor” itself predates the dired which I encountered in 1983–1984. Henry Bowlden told me about a new tape which the computing center had received from the University of Maryland (perhaps the one with DPS, since this conversation would have been in late 1979 or early 1980). He said that it had a lot of interesting programs including a “directory editor.” I asked what that was, but all I got from the explanation was that one could edit directories. Perhaps this page conveys the idea.
The computing center, by the way, used either a Univac 1100/20 or 1100/80 system around that time. In response to my question about being able to run programs received in that way, Henry said that lots of academics used Univacs (that IBM and DEC did not have all of that market). In between that initial discussion and seeing dired on a 4.2BSD system (1980-1983), I used the IBM VM/SP CM flist program, so the concept was not new to me.
Regarding terminology, there is more than one name:
“director-editor” is the oldest that I have encountered. As discussed in Dating DIRED, the term was used by a few developers for different programs during the early/mid-1970s.
“file-list” such as flist came from the IBM environment around the same time, e.g., mid/late-1970s.
The IBM system on which I used this did not have a hierarchical directory structure. Instead, it had a set of “minidisks” which were collections of files. With no hierarchy, there is no directory of interest, but a list of files. I suppose someone could have called it a disk-editor, but agree that would have a different connotation.
As of 2021, IBM still has documentation for the original flist as well as for the look-alike filelist (xedit script) which was introduced later, in the early 1980s.
“file-manager” came later, toward the end of the 1980s. The academics at the Software Productivity Consortium preferred that term, likely influenced by the Xerox Star which had more than one type of manager, e.g., desktop, window and file.
Not all were academics. In late 1987, I interviewed someone for a development position, who (seeing ded on my screen), immediately recognized it as a directory editor.
Most of ded's design dates from 1988, with more recent work being directed to
The line-drawing, color and other curses features (such as the management of the non-curses area of the screen) were why I became involved in ncurses, and provided the motivation to make ncurses more than the decaying hack which it was in early 1995.
I developed ded (and associated programs in cm_tools) to manage files stored in RCS/SCCS archives. Neither tool provided a way to tell which files were modified. It was a rare feature until the more elaborate IDEs were produced sometime after 2000. I built that into these tools, starting in 1986 with SCCS and in 1988 for RCS. Later, I added support for CVS and SVN (mainly to inspect files maintained by other developers — I had also written wrappers for the RCS/SCCS tools which suit my needs better than those other tools).
I use ded for both development and system maintenance. ded and vile are the first programs that I install on a UNIX-style development machine.
This is why:
As noted, working on ded was the reason why I became involved with ncurses. I was told about SIGWINCH in 1988, which was a feature of SunOS. That was a good idea, but curses implementations did not support it. I found that I could reallocate the memory used in BSD curses, and make ded respond to it, since that implementation used a simple scheme for allocating window data. But BSD curses did not use the line-drawing characters. SystemV curses did. But making SystemV curses work with SIGWINCH was a different matter. I found that reallocating a window was not straightforward due to the convoluted approach which it used, grouping similar-sized things into single chunks.
If there had been a way to communicate with the developers of SystemV curses and make them interested in the problem, that would have solved my problem. There was no apparent way to do that, so I put the idea aside for a few years until I noticed an early release of ncurses (which claimed that it was 100% compatible with SYSV curses, etc. — it was not).
Shortly after, I found the opportunity to send changes to its developers, starting with fixing the most apparent bugs. My goal was to get their implementation to be good enough that it could be used to interest the more established AT&T developers to use the idea. As it happens, both groups of developers fell by the wayside. I've been developing and maintaining ncurses since April 1996. ncurses does work with SIGWINCH.
Another aspect of ded influenced ncurses. That is the use of default colors.
Most curses applications fill the background with some color. Midnight Commander is a well-known example of this. I had used the Elvis text editor in MS-DOS; it used (like vile) a blue background. I used the same scheme for my adding machine.
However, ded divides the display into two parts:
The working area was originally inspired by dired. I extended that, making the working area resizable. With some care, it is possible to mix curses updates to the screen with shell commands that modify the working area. Some commands require ded to repaint the screen. There are corresponding commands for doing that.
One big problem was the use of color. Making the file-listing have a filled background would look ugly when combined with the plain working area. The color model used for the Linux console gave a solution to that: ded would set the terminal's background color to its default value; curses would be told how to manage text coloring on top of this.
SYSV curses did not do anything like this. But I had already (in 1995) become involved with ncurses, working to prop it up, to prepare it for SIGWINCH support. ncurses colors did not at that time work properly on Linux console and similar terminals which used the SGR 39/49 codes for default coloring. I pointed this out to ESR. After some resistance (which I overcame through constructing test programs to prove what SYSV curses did), he implemented a first version of background color support. That evolved (mostly by other developers) over the next 2-3 years.
Besides getting the colors right, there were other issues
between ncurses and ded. For instance, I had
found several years before that I could make repainting of the
work-area reliable by directly manipulating
curscr.
From dickey Fri Mar 31 06:23:06 1995 Subject: refresh-speed? To: esr@snark.thyrsus.com, esr@locke.ccil.org Date: Fri, 31 Mar 1995 06:23:06 -0500 (EST) X-Mailer: ELM [version 2.4 PL24alpha3] MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Content-Length: 804 Status: RO I noticed that the refresh-rate in xterm looked a little slow, so I set up a timing comparison between my 1.8.7 and 1.9 (directory-editor) binaries. What I tested: my editor uses only stdscr/curscr (because it manages a work area for shell commands, which can vary in size). So I'm not using scrolling-regions directly (because it's bsd/sysv compatible). I've got a directory-tree that can fake-scroll (refresh with lines 3-LINES updated up/down one line at a time). On a fast display, it's acceptable. I scripted my program through a few thousand up/down refreshes, and timed it. The 1.9 binary runs about 20% slower. I'd thought (from reading the notes) that the rewritten update/refresh code was looking for situations like this, to use scrolling regions? -- Thomas E. Dickey dickey@clark.net
Making ded work properly also included
forcing a window-repaint by refreshing curscr (a
feature which Eric Raymond was reluctant to modify in ncurses).
Interestingly enough, ncurses (see my notes
regarding Solaris in 1994) had this to say:
If the win argument to
wrefreshis the global variablecurscr, the screen is immediately cleared and repainted from scratch.
Late in 1995, I noticed the announcement that XFree86 had implemented color in xterm. (That was based on an older patch for color_xterm, which had been improved by two of the XFree86 developers). I took a look at that, and found that it did not implement the Linux-style default colors. So I proposed changes to improve xterm, mainly aimed at making its colors work as I wanted for ded. I did that, and have been continuing to develop and maintain xterm since January 1996.
As with ncurses, it took several months before I was "done" making the colors work properly. Early in November, I happened to be exchanging email with John Davis (developer of s-lang), and mentioned my plans for using the default colors in ncurses. He stated that he would implement support in s-lang, based on environment variables. My response was that I thought the design would be better if it were explicitly controlled by the application, rather than allowing environment variables to modify it in unforeseen ways. While s-lang implemented something first (about a month later), the changes that I put into ncurses early in 1997 were in my plan for it since I had started work on xterm.
There are two functions in ncurses which implement this design:
I persuaded one of the mutt (mail client) developers (Liviu Daia) to use default colors (August 1997), and implemented it in tin (September 1997) and lynx (December 1997). But using it in ded is the reason why it exists in ncurses.
I was recently (late 2013) reminded of this for another
reason. See FreeBSD #176871.
The report asked to remove the reference to ded
from default_colors(3x). There are a few pages outside
ncurses listed in the manpage SEE ALSO sections. Other than
a secondary issue in Debian #618429
there have been very few comments on these. To improve the
presentation on this website, I added a feature to
man2html to omit the HREF's from links outside the
ncurses directory. In doing that, I saw there are other programs
which one might suppose would attract more attention, such as
rogue (not a part of any well-managed
server).
The maintainer closed the FreeBSD report noting that they generally do not modify files from third-party repositories. There was an earlier report in Debian #295083 which had a different slant: remove the mention of ded because it was not packaged in Debian. Oddly, the requestor on this had already submitted an intention to package ded about six months before. The reader may notice the rude comment made late in the report by another person, and take into account his proposal made during the previous October, attempting to bypass Debian policy on x-terminal-emulator and undermine my work on xterm. However, the Debian package maintainer suggested a constructive resolution: I summarized the reason for the cross-reference.
ded uses line-drawing in its directory-tree. When I first wrote it, I could not count on having usable line-drawing. That was one of the reasons that I became interested in ncurses. Oddly, getting colors to work was less troublesome than getting line-drawing to work reliably. ded still has a command-line option to suppress the feature. My 1995 check-in comment says that it is "temporary".
On the other hand, ded's directory tree had little influence from other programs, nor have I seen it reflected in others. As noted, the concept of a directory-tree was suggested before xtree.
I saw xtree briefly in use in 1987, but did not use the IBM PC computer. A technician used that machine for burning PROMs. At the time, I was doing software development with SVr2 workstations. The concept was interesting, but there was a drawback in applying it to Unix: the PC program could read the whole directory (and from my recollection of a filesystem audit program which I wrote in the early 1990s, this could be done efficiently). My system had more than a hundred times as many files, which would require scanning a multitude of directories. On a typical system today, I will have more than 20 thousand directories (there are 44666 on the machine as I write this).
The obvious solution was to cache the directory information, updating it as needed. Perhaps not so obvious: the later program wcd did not do this type of caching when it was announced in May 1998. I recall wcd taking several minutes to start, because it built a list of all of the directories on my machine. ded builds its cache as it visits each new directory, and removes directories which are no longer found. I wrote the caching module for ded starting in September 1987. That was the last feature added to the interim program.
Other programs such as w3m and mc (Midnight Commander) support directory trees, but their approach to caching results is different from ded: less information is retained.
ded caches data to improve performance. It also allows you to decide whether the screen should be repainted after running a shell command, etc. This may have been more important in the late 1980s when I started writing it. Here are a few examples:
I asked Gabe to show me what he was working on. He did that. Then I asked if he'd like to see my program. He did, I did.
At the end, we were both unhappy: Gabe's program looked much nicer on the screen. Mine ran a hundred times faster.
The dired mode was very inefficient: I could hear my disk rattling for a minute or two as Steve wandered through my disk. If he had used ded, it would have taken only a second or so.
ded's directory tree supports folding (showing or hiding directory information). Each directory in the display can be folded independently. The commands for showing/hiding data also tell ded how deep to perform the folding, making it simple to display the first few levels of a deep directory tree.
I do not see this in other programs. They generally follow the scheme of Norton Commander:
Navigating in ded's directory tree differs from w3m and mc. In those programs, one can move the cursor up and down to select a different directory. Moving it left or right does not change the behavior.
In ded, moving the cursor left or right moves it to a different vertical line (associated with a different directory level). Once on a different vertical line, moving the cursor up or down moves within the selected directory level. This allows quick movement in a large tree.
All of this was before xterm. ded has supported xterm mouse-clicks since 1993.
ded has two displays:
It maintains one or more file-lists, which are stored as a
circularly-linked list. From a given file-list, you can refer to
the previous or next directory in shell commands (as
%B or %F), as well as paging to the
previous or next directory.
The file-lists are managed differently from say, Midnight
Commander. If I pipe a list of files using find to
ded, it determines the common prefix across the
items in the list and uses it as its working directory. Any list
can be further (temporarily) filtered using regular expressions.
If ded is run on a single filename, it uses a
regular expression to represent that, which can be
modified.
In mid-March 1996, I exchanged emails with Miguel de Icaza regarding ncurses (and slang). I mentioned ded, and he asked if I minded him looking at it for ideas. I replied
> > (I have a directory editor which is different in design from MC, which I'm
> > keeping curses-based).
>
> Is it free? May I take a look at it? Maybe I could merge some of
> your ideas into MC, I have busy schedule, but new ideas are always a
> good thing.
It's free. I've not advertised it much for a variety of reasons. (And it
won't ever look/act like MC, because some of the differences are part of the
design goals). Basically, I'm holding off on wider stuff til (if) I get my
bugs with ncurses resolved (longer story), and have time to implement
rebindable keys.
You can get a copy at
ftp.clark.net:/pub/dickey/ded
In my response I had in mind particularly the aspect of file-list management, and also default colors. Midnight Commander still uses a solid colored background.
As I mentioned above, ded accepts a list of pathnames via a pipe. I added this feature in 1991. Though it seems obvious, midnight commander still does not do it in 2014. I added the feature to use the results from checkup and conflict. After doing that, it later occurred to me that the results from find (and still later) locate could be piped into ded.
If it is obvious, why do some directory editors do this, and others do not? Reflecting, I recall reading that the IBM flist could read a list of files. Rereading the documentation here (the flist program) and here (flist "input areas"), it appears that the use of FIFOs in IBM flist was confined to the way it sent commands to the underlying control processor (CP). It did not provide a way to read a list of filenames (whether by file or FIFO) which would be displayed on the screen.
In either case, such a feature was not portable to VMS, and is not a feature of my version of flist.
Because ded uses default colors its appearance depends on the terminal. Here are examples in black and white, using xterm:
|
|
|
Here are examples showing the directory tree, with and without folding:
|
|
Download via ftp (you'll need the library):
Related utilities (RCS and SCCS):
Unrelated utilities (using td_lib):
Download via http (you'll need the library):
Related utilities (RCS and SCCS):
Unrelated utilities (using td_lib):
I have noticed these:
I build packages for my own use which you may find in the archive area:
There is no official Debian package (see discussion in default colors section), nor Red Hat.
These are, notwithstanding the comment about ded, descendents of the same dired program which I used in 1983-1984:
Each (including the 2.00 version from 1995) mentions “ded” in exactly one place (the makefile) which says
# PROGRAM = name of installed program (because some sites # call this program ded instead of dired)
That refers to Lepreau's version of dired, e.g., this snapshot. As with flist, I may have decided to use this name because it was less likely to be confused with dired, but was still related.