http://invisible-island.net/personal/
Copyright © 2013-2017,2019 by Thomas E. Dickey

Background

The sharutils program is interesting to me in more than one way:

Shell archives were used for distributing sets of source-files or patches via email or newsgroup postings. As such, this meant that they were used on Unix systems; other uses were minor niches. As of 2013, I still provide patches for some of the programs I work on, e.g., lynx, ncurses, vile, xterm. But for distributing sets of files, there are better ways.

History

I came into this story late, so some of this is necessarily secondhand. According to the sharutils documentation, it was based on work done in several steps:

The documentation also mentions uuencode and uudecode and work done on those by Ian Lance Taylor and Ulrich Drepper.

Some additional background is needed:

Looking for clues, I see this link which tells me that James Gosling thinks he wrote shar in 1978 or 1979. It shows a very simple script:

# shar -- Shell archiver
AR=$1
shift
for i do
        echo a - $i
        echo "echo x - $i" >>$AR
        echo "cat >$i <<'!Funky!Stuff!'" >>$AR
        cat $i >>$AR
        echo "!Funky!Stuff!" >>$AR
done

The script constructs here-documents which rely on the embedded files not containing special characters (such as newline) or the magic string !Funky!Stuff!. Subsequent implementations by others addressed these limitations, e.g., using sed-scripts to apply a "X" prefix to each line of the file, or uuencoding the file.

The 1978/1979 date is a little nebulous, considering that the comment was made in 2006 — 25-30 years later. There is a cshar program in the DECUS archive dated December 1984 by Michael A. Thompson which gives the same information in its header comment:

/*
Shar puts readable text files together in a package
from which they are easy to extract.  The original version
was a shell script posted to the net, shown below:
        #Date: Mon Oct 18 11:08:34 1982
        #From: decvax!microsof!uw-beave!jim (James Gosling at CMU)
        AR=$1
        shift
        for i do
                echo a - $i
                echo "echo x - $i" >>$AR
                echo "cat >$i <<'!Funky!Stuff!'" >>$AR
                cat $i >>$AR
                echo "!Funky!Stuff!" >>$AR
        done
I rewrote this version in C to provide better diagnostics
and to run faster.  The main difference is that my version
does not affect any files because it prints to the standard
output.  Mine also has a -v (verbose) option.
*/

Gosling, incidentally, is well-known for Emacs and Java.

William Davidsen was active in Usenet (among other things, a moderator for the Intel 80386 topics) during the late 1980s until the early 1990s. Like me, Davidsen was a beta tester for mawk 1.2, and is mentioned in the credits for tin. Davidsen posted shar2 aka xshar to comp.sources.misc in May, 1988 (volume 3). It was not the first shar program posted; Brandon S. Allbery posted mkshar in January 1988 (volume 2). Shortly after shar2, Gail Zacharias posted perlshar.

There were two more postings of shar-related programs to comp.sources.misc in 1989 (unshar.perl and unshar.uemacs—a MicroEmacs macro).

Oddly, Davidsen used shar 1.27 when posting ddf to comp.sources.misc in 1990. (Without studying the shell archives, it is hard to see which program generated them; most early versions of shar did not print a version number, e.g., this example from 1987.

Warren Tucker posted his changes to alt.sources:

Richard H. Gumpertz continued, releasing shar 3.49 in August 1990 (see for example this posting using 3.49). The "3.49" appears to be the RCS version of one or more parts of the program, as seen in this old change-log. Some changes were made after "3.49", though.

Other Programs

What about cshar? It was distributed via comp.sources.unix, as indicated here. The note comments that shar 3.49 works with uuencode/uudecode, while cshar's makekit handles splitting and kit-building.

There appears to have been more than one cshar:

comp.sources.unix died in 1996 or 1997 and after almost ten years was removed. Incidentally, one of the last programs to be posted on it was Cdk 4.7.0, in May 1996 (I maintain Cdk). Likewise, my program conflict was one of the last. to be posted on comp.sources.misc, in April 1995. The last program posted on comp.sources.reviewed was csize, in 1994 (my review, one of five submitted, was the shortest, and alluded to c_count).

My Involvement

While I had access to Usenet from the late 1980s on, and obtained several programs by unshar'ing them from newsgroup postings I did not become interested in creating my own shar files until I left the Software Productivity Consortium early in 1994. From late 1992, I had switched direction, making improvements to programs started by others, rather than developing entirely new programs as I had done to support the older projects from the 1980s and before. In my newer projects, I wanted to continue developing programs such as vile and use those which I had written by myself (in particular, ded) while avoiding conflict-of-interest issues. The way to do this, I decided, was to ensure that my external projects were clearly visible.

Maintaining an ftp site did not meet that need; it was not visible enough. Nor did uploading to a common ftp site appear to be an improvement. I decided that the source-newsgroups which were archived on ftp sites would be a good solution.

Though I had contributed fixes for a few programs which were distributed as shar-files (see this discussion), I had not done this for my own programs. I discussed this with Paul Fox (original developer of vile) in July 1994 (shortly after I submitted diffstat to comp.sources.unix).

Around the same time, I sent a fix for the shar 3.52 beta:

   I've a patch which makes the 'touch' option work properly on
   machines that haven't got a 'touch' command that recognizes the
   timestamp option (e.g., SunOS 4.1.x).

The patch provided C code for a replacement program. François Pinard replied with a further refinement, and thereafter I was on the mailing list for the succession of beta versions. This also led me into involvement with autoconf betas before its 2.0 release.

As one may see from the change-log, my involvement was at the beginning, up until it was renamed to sharutils and released as 4.1, etc. After that, I was busy with other projects. My last mail from the sharutils mailing list was in December 1996.

In short, I am one of about 60 people who have contributed to sharutils:

Oddly (despite its lessened usage), sharutils has grown (counting bundled libraries) more than ten times larger than the release which I worked on. The table shows C SLOCs (using c_count) and the size of the unpacked source tar-file:

Date Version SLOCs Size (Mb)
1994/11/05 4.1 6606 0.66
1995/12/24 4.2 10229 1.42
2005/04/24 4.3.79 23879 3.88
2005/07/07 4.4 23943 3.81
2005/08/13 4.5.1 23901 3.84
2005/09/09 4.6 23907 3.89
2006/08/30 4.7 23827 3.98
2010/02/19 4.8 47344 6.08
2010/02/21 4.9 49082 6.17
2010/08/27 4.10 49071 6.12
2011/02/05 4.12 51972 6.32
2012/12/22 4.13 82846 8.27
2013/04/13 4.14 93615 10.66

Obsolete?

These programs are rarely used now. Some do not use sharutils (arguably the most complete implementation).

For instance this 2006 FreeBSD review uses an old version of shar (a shell script), as one can see by examining the script from the base system (e.g., release 10). Oddly, the script is not much more than an improved version of the script attributed to James Gosling. It does not use uuencoding (for handling non-text files), although the technique is said to be very old, e.g., from 4.0BSD:

(both written by Mark Horton in 1980, and posted to mod.sources).

NewsGroup Archives