http://invisible-island.net/
Copyright © 1996-2013,2014 Thomas E. Dickey


DED – Directory-Editor

Synopsis

ded (pronounced "dee–eee–dee") allows you to navigate through multiple file lists or a directory tree, viewing or changing file attributes rapidly. In addition to conventional file information, it operates on the file's RCS or SCCS archives (or CVS or SVN, etc), making it useful for source-control as well as system administration. Curses-based, it runs on UNIX systems.

History

This is a long-term project. I originally began in 1984, enhancing a version of dired (a program in C, written by Jay Lepreau <lepreau@cs.utah.edu>) while at the ITT Advanced Technology Center. There is some dispute over whether that dired is based on the emacs mode introduced around the same time (1980) or the reverse (since neither program's documentation at the time credited the other), but it is indisputable that both were inspired by an earlier stand-alone program running on Tenex available in the Stanford AI Lab (SAIL) in 1978.

That early dired had crude display management, which allowed a user to enter shell commands and see them at the bottom of the screen—not something that curses was designed to do. Among other changes, early in 1984, I modified the program to allow the user to change the fraction of the screen which was devoted to the command output. Around that time (actually October 1984, after I was not actively working on dired and had moved to the project where I wrote flist), an associate asked if I was planning to add a directory-tree display. This was about six months before xtree was introduced; he had apparently seen some earlier program with the feature but was unable to provide details.

There are later versions of dired. however I found nothing useful in the later changes since the most apparent changes were more complicated (and cumbersome) key-binding for the functions.

Subsequently, I designed and implemented a series of directory editors including flist, and this program (which uses a directory-tree module, written in 1987, from an earlier program begun at the end of 1985). By then I had seen xtree (but not used it—the designs are different).

Although dired used BSD curses, my intervening programs did not. flist uses VMS's SMG$ calls. The program for which I originally wrote the directory-tree module used a wrapper for termcap which I wrote when I found that the curses library on the Sperry PC (late 1985) did not work. The wrapper (like the one which I wrote for tin in the mid-1990s) provides an interface using curses's function names, but like other low-level interfaces (such as conio and s-lang) did not provide optimization.

The term “directory editor” itself predates the dired which I encountered in 1983–1984. Henry Bowlden told me about a new tape which the computing center had received from the University of Maryland (perhaps the one with DPS, since this conversation would have been in late 1979 or early 1980). He said that it had a lot of interesting programs including a “directory editor.” I asked what that was, but all I got from the explanation was that one could edit directories. Perhaps this page conveys the idea.

The computing center, by the way, used either a Univac 1100/20 or 1100/80 system around that time. In response to my question about being able to run programs received in that way, Henry said that lots of academics used Univacs (that IBM and DEC did not have all of that market).

Most of ded's design dates from 1988, with more recent work being directed to

The line-drawing, color and other curses features (such as the management of the non-curses area of the screen) were why I became involved in ncurses, and provided the motivation to make ncurses more than the decaying hack which it was in early 1995.


Features

I developed ded (and associated programs in cm_tools) to manage files stored in RCS/SCCS archives. Neither tool provided a way to tell which files were modified. It was a rare feature until the more elaborate IDEs were produced sometime after 2000. I built that into these tools, starting in 1986 with SCCS and in 1988 for RCS. Later, I added support for CVS and SVN (mainly to inspect files maintained by other developers — I had also written wrappers for the RCS/SCCS tools which suit my needs better than those other tools).

I use ded for both development and system maintenance. ded and vile are the first programs that I install on a UNIX-style development machine.

This is why:

Influence

SIGWINCH — Resizing Windows

As noted, working on ded was the reason why I became involved with ncurses. I was told about SIGWINCH in 1988, which was a feature of SunOS. That was a good idea, but curses implementations did not support it. I found that I could reallocate the memory used in BSD curses, and make ded respond to it, since that implementation used a simple scheme for allocating window data. But BSD curses did not use the line-drawing characters. SystemV curses did. But making SystemV curses work with SIGWINCH was a different matter. I found that reallocating a window was not straightforward due to the convoluted approach which it used, grouping similar-sized things into single chunks.

If there had been a way to communicate with the developers of SystemV curses and make them interested in the problem, that would have solved my problem. There was no apparent way to do that, so I put the idea aside for a few years until I noticed an early release of ncurses (which claimed that it was 100% compatible with SYSV curses, etc. — it was not).

Shortly after, I found the opportunity to send changes to its developers, starting with fixing the most apparent bugs. My goal was to get their implementation to be good enough that it could be used to interest the more established AT&T developers to use the idea. As it happens, both groups of developers fell by the wayside. I've been developing and maintaining ncurses since April 1996. ncurses does work with SIGWINCH.

Default Colors

Another aspect of ded influenced ncurses. That is the use of default colors.

Most curses applications fill the background with some color. Midnight Commander is a well-known example of this. I had used the Elvis text editor in MS-DOS; it used (like vile) a blue background. I used the same scheme for my adding machine.

However, ded divides the display into two parts:

The working area was originally inspired by dired. I extended that, making the working area resizable. With some care, it is possible to mix curses updates to the screen with shell commands that modify the working area. Some commands require ded to repaint the screen. There are corresponding commands for doing that.

One big problem was the use of color. Making the file-listing have a filled background would look ugly when combined with the plain working area. The color model used for the Linux console gave a solution to that: ded would set the terminal's background color to its default value; curses would be told how to manage text coloring on top of this.

SYSV curses did not do anything like this. But I had already (in 1995) become involved with ncurses, working to prop it up, to prepare it for SIGWINCH support. ncurses colors did not at that time work properly on Linux console and similar terminals which used the SGR 39/49 codes for default coloring. I pointed this out to ESR. After some resistance (which I overcame through constructing test programs to prove what SYSV curses did), he implemented a first version of background color support. That evolved (mostly by other developers) over the next 2-3 years.

Late in 1995, I noticed the announcement that XFree86 had implemented color in xterm. (That was based on an older patch for color_xterm, which had been improved by two of the XFree86 developers). I took a look at that, and found that it did not implement the Linux-style default colors. So I proposed changes to improve xterm, mainly aimed at making its colors work as I wanted for ded. I did that, and have been continuing to develop and maintain xterm since January 1996.

As with ncurses, it took several months before I was "done" making the colors work properly. Early in November, I happened to be exchanging email with John Davis (developer of s-lang), and mentioned my plans for using the default colors in ncurses. He stated that he would implement support in s-lang, based on environment variables. My response was that I thought the design would be better if it were explicitly controlled by the application, rather than allowing environment variables to modify it in unforeseen ways. While s-lang implemented something first (about a month later), the changes that I put into ncurses early in 1997 were in my plan for it since I had started work on xterm.

There are two functions in ncurses which implement this design:

I persuaded one of the mutt (mail client) developers (Liviu Daia) to use default colors (August 1997), and implemented it in tin (September 1997) and lynx (December 1997). But using it in ded is the reason why it exists in ncurses.

I was recently (late 2013) reminded of this for another reason. See FreeBSD #176871. The report asked to remove the reference to ded from default_colors(3x). There are a few pages outside ncurses listed in the manpage SEE ALSO sections. Other than a secondary issue in Debian #618429 there have been very few comments on these. To improve the presentation on this website, I added a feature to man2html to omit the HREF's from links outside the ncurses directory. In doing that, I saw there are other programs which one might suppose would attract more attention, such as rogue (not a part of any well-managed server).

The maintainer closed the FreeBSD report noting that they generally do not modify files from third-party repositories. There was an earlier report in Debian #295083 which had a different slant: remove the mention of ded because it was not packaged in Debian. Oddly, the requestor on this had already submitted an intention to package ded about six months before. The reader may notice the rude comment made late in the report by another person, and take into account his proposal made during the previous October, attempting to bypass Debian policy on x-terminal-emulator and undermine my work on xterm. However, the Debian package maintainer suggested a constructive resolution: I summarized the reason for the cross-reference.


Directory Tree

ded uses line-drawing in its directory-tree. When I first wrote it, I could not count on having usable line-drawing. That was one of the reasons that I became interested in ncurses. Oddly, getting colors to work was less troublesome than getting line-drawing to work reliably. ded still has a command-line option to suppress the feature. My 1995 check-in comment says that it is "temporary".

On the other hand, ded's directory tree had little influence from other programs, nor have I seen it reflected in others. As noted, the concept of a directory-tree was suggested before xtree.

I saw xtree briefly in use in 1987, but did not use the IBM PC computer. A technician used that machine for burning PROMs. At the time, I was doing software development with SVr2 workstations. The concept was interesting, but there was a drawback in applying it to Unix: the PC program could read the whole directory (and from my recollection of a filesystem audit program which I wrote in the early 1990s, this could be done efficiently). My system had more than a hundred times as many files, which would require scanning a multitude of directories. On a typical system today, I will have more than 20 thousand directories (there are 44666 on the machine as I write this).

Caching Directory Data

The obvious solution was to cache the directory information, updating it as needed. Perhaps not so obvious: the later program wcd did not do this type of caching when it was announced in May 1998. I recall wcd taking several minutes to start, because it built a list of all of the directories on my machine. ded builds its cache as it visits each new directory, and removes directories which are no longer found. I wrote the caching module for ded starting in September 1987. That was the last feature added to the interim program.

Other programs such as w3m and mc (Midnight Commander) support directory trees, but their approach to caching results is different from ded: less information is retained.

ded caches data to improve performance. It also allows you to decide whether the screen should be repainted after running a shell command, etc. This may have been more important in the late 1980s when I started writing it. Here are a few examples:

Hiding Directory Data

ded's directory tree supports folding (showing or hiding directory information). Each directory in the display can be folded independently. The commands for showing/hiding data also tell ded how deep to perform the folding, making it simple to display the first few levels of a deep directory tree.

I do not see this in other programs. They generally follow the scheme of Norton Commander:

Navigating in the Directory Tree

Navigating in ded's directory tree differs from w3m and mc. In those programs, one can move the cursor up and down to select a different directory. Moving it left or right does not change the behavior.

In ded, moving the cursor left or right moves it to a different vertical line (associated with a different directory level). Once on a different vertical line, moving the cursor up or down moves within the selected directory level. This allows quick movement in a large tree.

All of this was before xterm. ded has supported xterm mouse-clicks since 1993.

Navigating outside the Directory Tree

ded has two displays:

It maintains one or more file-lists, which are stored as a circularly-linked list. From a given file-list, you can refer to the previous or next directory in shell commands (as %B or %F), as well as paging to the previous or next directory.

The file-lists are managed differently from say, Midnight Commander. If I pipe a list of files using find to ded, it determines the common prefix across the items in the list and uses it as its working directory. Any list can be further (temporarily) filtered using regular expressions. If ded is run on a single filename, it uses a regular expression to represent that, which can be modified.

In mid-March 1996, I exchanged emails with Miguel de Icaza regarding ncurses (and slang). I mentioned ded, and he asked if I minded him looking at it for ideas. I replied

> > (I have a directory editor which is different in design from MC, which I'm
> > keeping curses-based).
> 
> Is it free?  May I take a look at it?  Maybe I could merge some of
> your ideas into MC, I have busy schedule, but new ideas are always a
> good thing.
It's free.  I've not advertised it much for a variety of reasons.  (And it
won't ever look/act like MC, because some of the differences are part of the
design goals).  Basically, I'm holding off on wider stuff til (if) I get my
bugs with ncurses resolved (longer story), and have time to implement
rebindable keys.

You can get a copy at

        ftp.clark.net:/pub/dickey/ded

In my response I had in mind particularly the aspect of file-list management, and also default colors. Midnight Commander still uses a solid colored background.

Screenshots

Because ded uses default colors its appearance depends on the terminal. Here are examples in black and white, using xterm:

Black background White background

Here are examples showing the directory tree, with and without folding:

With folding Without folding

Documentation


Download

Download via ftp (you'll need the library):

Related utilities (RCS and SCCS):

Unrelated utilities (using td_lib):


Download via http (you'll need the library):

Related utilities (RCS and SCCS):

Unrelated utilities (using td_lib):

Related Links

Packages

I have noticed these:

There was a ded package in SlackBuilds until release 13, which was maintained by slakmagik.

I build packages for my own use which you may find in the ftp area:

There is no official Debian package (see discussion in default colors section), nor Red Hat.

User comments

Unrelated programs