https://invisible-island.net/personal/
Copyright © 2018-2023,2024 by Thomas E. Dickey
Cross-compiling can aid in development for programs where the target platform has poor tool support. Tool support has several aspects: cost, availability and the suitability of a tool for its intended use.
Cross-compiling complements portability by enabling the use of development machines with better capabilities than a given target environment.
A portable program is one that has been made to work on multiple platforms:
When I began porting programs in the mid-1980s, there were only a few platforms available to me: SVr2 workstations, MS-DOS and Xenix.
A few years later, there were several flavors of Unix available, and VMS.
Still a few years later, there were OS/2, Windows 3.1, and Linux.
The available platforms influenced the nature of the programs to be considered for porting. Rather than porting a massive application (or suite of programs), it was useful to port simple tools which did not use many system-dependent features.
For example, I ported both c_count (in 1990) and cproto (in 1993) to VMS with minimal change:
provide a build-script, using MMS (VMS's equivalent to make).
provide missing functions i.e., getopt.
add a few #ifdef lines to cover the system differences, e.g., the lack of pipes (and popen) on VMS.
The MMS program was a problem, since it was an additional feature (not bundled with the C compiler) which not all VMS systems might have. The alternative (writing a brute-force build.com as I did for flist) was unsuitable for development. That problem was ultimately solved by MMK, Matthew Madison's clone of MMS in the mid-1990s (see current source).
In that regard, VMS was unusual. Build-environments on other platforms have provided a make program even when the tool developers would rather that their users use an IDE.
Simple tools are easily ported. Not all useful programs are simple. Consider vile, lynx, ncurses. With more complicated programs, which used more system-dependent features, there were compelling reasons to make these programs fully exploit the systems on which they ran. The ports for vile, lynx, and ultimately ncurses, use native I/O. For example, Windows provides platform-specific functions which can be used to support these tools:
Other platforms (e.g., OS/2, VMS) provided their own platform-specific functions for native I/O.
For some time, using these features tied development into tools which ran only on the target platform.
Early on, all of the ports which I worked on were developed on the target environment. During the 1980s, I worked on programs which I ported between different platforms, e.g.,
a meta assembler begun on VMS and ported to 4.2BSD Unix.
a kernel driver for a TCP/IP board, along with utilities, ported to a half-dozen SVr2 systems as well as Xenix.
Later, during the 1990s, I continued in that way, with a mixture of programs which I created and those which others had begun.
Vile development began in August 1990; I became involved in December 1992.
Vile uses separate files for terminal I/O drivers (and separate build scripts for different platforms). It also uses platform-dependent functions for handling file formats (e.g., VMS).
In addition to the *nix platforms with termcap/terminfo, curses and X11 drivers, these platforms have special I/O drivers:
Lynx development began shortly after my involvement with vile, though I did not become involved with Lynx until early 1997.
Lynx uses a simpler approach to displaying on the terminal (it usually uses just the curses API), but has platform-specific features for file I/O (VMS and MS-DOS pathnames) as well as networking features. Inevitably, there are platform-specific build-scripts.
According to comments in the source, the VMS port was begun by Lou Montulli in January 1993, using the CERN WWW library (developed 1990-1993). Foteos Macrides was active in that effort starting late in 1993. Macrides also made changes for Windows (console) ports in 1995, though others (Doug Kaufman and Wayne Buttles) appear to have done most of the development for the MS-DOS and Windows ports.
The changelog suggests that I began the OS/2 port in February 1998, though there was an earlier port by someone who did not provide source code (see mailing list from May 1997).
Either way, Lynx used the EMX (BSD-based) runtime libraries with OS/2 rather than native I/O.
MacRides stopped working on Lynx after releasing 2.7.2 at the end of 1997. I was able to continue work on the VMS port (using others' machines) until September 2008 (when the HP test-drive environment was no longer available).
I did most of the initial work to port ncurses to various Unix systems, but for most of the non-Unix ports, others began those, and I finished them. The development environments for each of these were similar, using gcc on machines which had ample memory and diskspace:
QNX, begun by Mike Hunter in 1996.
OS/2 (EMX), begun by J.J.G.Ripoll in 1997.
BeOS, begun in 1999, incorporating fixes by developers for related programs.
MinGW, begun in 2009 by Juergen Pfeiffer.
Though I had started a port in 2003, it was stalled by the problem of how to reuse the scripts which generate source-files during the build.
There was no VMS port for ncurses. Other developers started this, but did not produce a workable port, as noted in the ncurses FAQ.
Each of the workable ports relied upon a set of POSIX utilities for running the build-scripts, and a low-cost compiler.
The larger programs (vile, lynx, ncurses) have many features. Some of those features can be improved by using platform-specific features. Smaller/simpler programs are more readily ported. For instance, these required little more than providing a suitable makefile and reimplementations of functions such as getopt:
When I began developing C/C++ programs for MS-DOS and Windows 3.1 in 1993, I used TurboC++. The TurboC++ IDE was useful for debugging, but makefiles were the proper way to build vile (March 1993) and add (October 1993).
A little later (in 1995), developing on OS/2 Warp, I used the IBM C Set compiler. It had an IDE (see summary of IBM C Set 2.0 on WinWorld), but I did not find that useful.
Borland TurboC++ and IBM C Set cost money; vile and other programs which I develop outside my day job are distributed free of charge.
Paul Fox recommended djgpp, a port of gcc which ran on MS-DOS (using a memory extender), which cost nothing. After a while, I added that to the development tools which I used with MS-DOS.
For building a program, just the compiler and a build-tool (and libraries) are needed.
When I began developing vile on MS-DOS in 1993, there were a few choices, ordered here from no-cost to several hundred dollars:
In paid work, a year or two later, I used Watcom (versions 10 and 11) and Visual C++ (beginning with version 4.0), but did not purchase a Visual C++ license until I began to develop winvile in 1997. That was for Visual C++ 5.0, which became obsolete a year later by the introduction of Visual C++ 6.0, and later by Visual Studio 2003 and 2005. I used those as well, letting others pay for licenses.
Today (writing this in 2019), a usable version of Visual Studio can be obtained free of charge, but until Visual Studio Express 2008 was released, there was no low-cost version of Visual Studio available.
When developing for MS-DOS, I used the TurboC++ and Watcom IDEs in the early stages of a port, but relied on makefiles for most of the development work. The output from running make could be redirected to a file for analysis.
With different compilers, there was a problem: each of those compilers required a different set of environment variables to give the location of the compiler, the header files and the libraries. Compiling with DJGPP was simple. The DJGPP user's guide mentions just PATH and DJGPP. TurboC++ used three (PATH, INCLUDE, LIB). Watcom added to that list (see The WATCOM C/C++ Programmer's FAQ). Using more than one compiler on the same machine required an initialization script.
Developing with TurboC++ and Windows 3.1 did not work well because Windows 3.1 would stop while in TurboC++'s debugger. After Windows NT was available, I revisited this with a copy of Petzold's book and was able to get winvile working.
Visual C++ used even more environment variables than the preceding toolsets. It provided an initialization script (vcvars32.bat), to help. Calling the script was a nuisance because there was no way to run that automatically when starting the command-line interpreter cmd. There was a feature of cmd which looked as it it would solve that problem, but it did not work well enough:
It was possible to use cmd's /start command to start a new console window (after calling vcvars32.bat),
Using “cmd start” in this manner would leave the first window still running.
Closing the first window would stop the second window.
After several years, Microsoft improved this behavior, but during the early 2000s, this only provided motivation to look for a better solution. Rather than start another console window, I decided to run winvile:
The environment variables would work in winvile.
From winvile, I could run make, and use winvile's error-finder to walk through problems in source-files.
That console window was a problem. I found that it was usually (not always) possible to start a batch file via a shortcut “minimized” which would in turn start winvile. Since that did not always work (and killing the intermediate window would kill winvile), I looked for a different solution. I found one using the “Send To” feature of the Windows desktop:
After some discussion, Paul Fox added a feature in the command-language for vile to set environment variables (which would be inherited in the make process). I used this feature to translate vcvars32.bat into vile's scripting language.
I solved the problem of selecting the appropriate initialization file for winvile by making the shortcut start it in a special directory which read the definitions for a specific version of Visual C++.
The shortcut for “Send To” allowed passing a filename after the executable name. I changed winvile to switch to that file's directory on startup.
For example, here is the initialization script vile.rc for Visual C++ 4:
source c:/vile/vile.rc
source vc4.vile
The first “source” command uses the normal initialization script, makiing it reusable across different compilers. Here (after trimming project-specific stuff) is /vile/vile.rc:
source vileinit.rc
set fk=auto
set modeline
set backup-style=off
set fc=78
set swap
set cs=black
set printing-high=255
set autocolor=50
set vis=rev
set recent-files=20
set recent-folders=20
bind-key execute-hypertext-command ^X-z
bind-key show-hypertext-command ^X-Z
~with set-extra-colors
modeline reverse+green
string underline+magenta
regex magenta
number cyan
hypertext underline+red
enum green
isearch reverse+blue
warning reverse+bold+red
linenumber cyan+reverse
~endwith
store-procedure wingrep i="Row" i="Col"
~local %col
setv %col &sub $2 1
$1 goto-line
beginning-of-line
~if &ge %col 1
%col forward-character-to-eol
~endif
~endm
set modeline
Here is the translated vcvars32.bat script vc4.vile:
store-procedure prepend s="Variable" s="Directory"
~local %name %value
setenv $1 &cat $2 &cat $pathlist-separator &env $1
~endm
; begin vcvars32.bat
;
; Root of Visual C++ installed files.
setv %MSDevDir='g:\MSDEVSTD'
; VcOsDir is used to help create either a Windows 95 or Windows NT specific path.
~if &equ &env "OS" "Windows_NT"
setv %VcOsDir=WINNT
~else
setv %VcOsDir=WIN95
~endif
write-message 'Setting environment for using Microsoft Visual C++ tools.'
; Include cdrom files in environment.
setv %vcsource=''
~if ¬ &equ %vcsource ''
~if &or &equ &env "OS" "Windows_NT" \
&equ &env "OS" ""
prepend "PATH" &cat %vcsource &cat '\BIN\' %VcOsDir
prepend "PATH" &cat %vcsource '\BIN'
~endif
prepend "INCLUDE" &cat %vcsource '\MFC\INCLUDE'
prepend "INCLUDE" &cat %vcsource '\INCLUDE'
prepend "LIB" &cat %vcsource '\MFC\LIB'
prepend "LIB" &cat %vcsource '\LIB'
~endif
~if &or &equ &env "OS" "Windows_NT" \
&equ &env "OS" ""
prepend "PATH" &cat %MSDevDir &cat '\BIN\' %VcOsDir
prepend "PATH" &cat %MSDevDir '\BIN'
~endif
prepend "INCLUDE" &cat %MSDevDir '\MFC\INCLUDE'
prepend "INCLUDE" &cat %MSDevDir '\INCLUDE'
prepend "LIB" &cat %MSDevDir '\MFC\LIB'
prepend "LIB" &cat %MSDevDir '\LIB'
; end vcvars32.bat
Initially, I did not save a copy of the original Microsoft script, but after a few years decided it would be useful because Microsoft was changing the way the script was used. Here is the original script for Visual C++ 6:
@echo off
rem
rem Root of Visual Developer Studio Common files.
set VSCommonDir=G:\PROGRA~1\MICROS~1\Common
rem
rem Root of Visual Developer Studio installed files.
rem
set MSDevDir=G:\PROGRA~1\MICROS~1\Common\msdev98
rem
rem Root of Visual C++ installed files.
rem
set MSVCDir=G:\PROGRA~1\MICROS~1\VC98
rem
rem VcOsDir is used to help create either a Windows 95 or Windows NT specific path.
rem
set VcOsDir=WIN95
if "%OS%" == "Windows_NT" set VcOsDir=WINNT
rem
echo Setting environment for using Microsoft Visual C++ tools.
rem
if "%OS%" == "Windows_NT" set PATH=%MSDevDir%\BIN;%MSVCDir%\BIN;%VSCommonDir%\TOOLS\%VcOsDir%;%VSCommonDir%\TOOLS;%PATH%
if "%OS%" == "" set PATH="%MSDevDir%\BIN";"%MSVCDir%\BIN";"%VSCommonDir%\TOOLS\%VcOsDir%";"%VSCommonDir%\TOOLS";"%windir%\SYSTEM";"%PATH%"
set INCLUDE=%MSVCDir%\ATL\INCLUDE;%MSVCDir%\INCLUDE;%MSVCDir%\MFC\INCLUDE;%INCLUDE%
set LIB=%MSVCDir%\LIB;%MSVCDir%\MFC\LIB;%LIB%
set VcOsDir=
set VSCommonDir=
In Visual C++ 7 (Visual Studio 2003), Microsoft renamed that to vsvars32.bat, but kept a vcvars32.bat (which called vsvars32.bat), for compatibility.
I translated vcvars32.bat for several versions (4, 4.1, 5, 6 and 7) before Microsoft's newer Visual Studio provided a far more complex script to address different architectures, e.g., 32-bit and 64-bit machines. Starting with Visual Studio Express 2008, I used the original approach, which (with Windows XP) largely solved the problem of the stray console window. Here is an example:
@echo off
setlocal
set PARTITION=vs2008x32
set TARGET_ARCH=i386
set TARGET_ARCH=x86
rem trim path to get rid of cygwin
set PATH=C:\Windows\system32;C:\Windows;c:\com
rem add PERL for vile's builds
set PATH=%PATH%;c:\perl\bin
call "c:\program files (x86)\microsoft visual studio 9.0\vc/vcvarsall.bat" %1
shift
run-winvile %*
endlocal
The second script, run-winvile.bat, handles differences in the winvile configuration:
@echo off
setlocal
set WINVILE_DIR="C:\Program Files (x86)\VI Like Emacs\bin"
cd %WINVILE_DIR%
IF ERRORLEVEL 1 goto :finish
set WINVILE_EXE=WinVile-wide-ole.exe
if exist %WINVILE_EXE% start %WINVILE_EXE% %1 %2 %3 %4 %5 %6 %7 %8 %9
:finish
endlocal
Those use nmake, which though it has become less popular, has the advantage that periodic updates for Visual Studio do not break the makefiles. That cannot be said of the Visual Studio project files, nor the “modern” VCBuild and MSBuild tools which have cropped up in the Visual Studio releases.
Although it would seem to follow naturally after DJGPP, I did not consider developing with MinGW as an option until the mid-2000s. My email reminds me that other developers (initially for XFree86 and lynx) were involved in ports using MinGW as early as 1999. As the maintainer for Lynx, I integrated changes by a few others.
Other compilers that I used on Windows include
That ignores cygwin and msys2; both are environments which can support tools for cross-compiling to use the Microsoft libraries.
Because setting up development machines took substantial time, and porting programs involved several changes, I found it helpful to maintain source-control archives on the development machines. Doing that let me record changes as I made them, and use that information in updating the archives on my main machine.
During the 1990s, nothing appeared suitable for this purpose.
After 2000, things changed. I used ComponentSoftware's CS-RCS (from 2001-2005). This worked as an add-on to the Windows Explorer which executed RCS utilities that ran under a separate process (to allow making the front end add-on proprietary code as discussed in this thread). The reason for the separate process was to avoid entanglement with the dynamic linking issue.
Ultimately, I stopped using CS-RCS because it interfered with Windows Explorer when logged in as a user other than the one registered to use the product.
As a replacement, I used CVSNT in 2006, finding its local server awkward to use. At the same time, I tried TortoiseSVN and Perforce.
After a while, TortoiseGit was available. I use it to track updates to locally-built libraries (better than CS-RCS). Where CS-RCS was good (tracking changes to stray configuration files), Git is far less flexible.
Newer development machines have better networking support than the machines I used around 2000. I use WinSCP for mirroring development sources and logfiles to my file server.
Cross-compiling has the potential advantage of using tools and resources which may be more powerful than those on the target machine. It has the drawback that the cross-compiler itself may be less reliable, as well as that the libraries used in cross-compiling may not be up to date.
That said, the potential advantages seem to outweigh the drawbacks.
Building is more complicated. Cross-compiling involves more than just changing the name of the tool. In some programs, the configuration has to distinguish between the cross-compiler itself and tools which are built as part of the build. For example:
vile uses a table-generator mktbls
lynx uses a table-generator makeuctb
ncurses uses a table-generator make_hash.
However, ncurses also relies upon awk and sed scripts for generating source-files. That makes porting ncurses to a non-POSIX development environment (such as Windows or VMS) more difficult than the other programs.
All of these use autoconf-based configure scripts. The table-generators are necessarily very simple and portable because the autoconf checks are not useful for those programs. That is because autoconf handles only one compile-environment. Table generators require two.
At the time (and perhaps still), the autoconf developers did not get involved in solving this problem. They were more concerned about terminology, as you can see by reading the documentation. As a result, the terminology which I use in my configuration scripts may differ (see CF_BUILD_CC for instance, which I began in 2004, from my work on ncurses beginning in 1998).
The DJGPP website has instructions for building a cross-compiler:
http://www.delorie.com/howto/djgpp/linux-x-djgpp-revised.html
Linux x86 Host Cross Compiler HOWTO for the DJGPP Target Revised
Charles Wilkins (November 14, 2002).
I followed those instructions and was able to set up a cross compiler for DJGPP in September 2003. In addition to the generic script cfg-djgpp
#!/bin/sh
# Id: cfg-djgpp,v 1.4 2010/05/22 17:32:43 tom Exp # configure to cross-compile for djgcc
#
# use: testing-9 (gcc 3.3)
# use: xen-freebsd (gcc 4.1.0)
#
TARGET=i586-pc-msdosdjgpp
TOOLS=/usr/local/compiler/cross/djgpp
if test -d $TOOLS
then
export PATH=$TOOLS:$PATH
fi
cfg-normal \
--with-build-cc=gcc \
--host=$TARGET \
--target=$TARGET \
$*
which I mentioned on the ncurses mailing list in 2006, I also created customized versions for lynx
#!/bin/sh
# configure lynx cross-compiled to djgpp
DJAPPS=/usr/local/djgpp
WTAPPS=$DJAPPS/contrib/watt32
export CPPFLAGS="-I$WTAPPS/inc -I$DJAPPS/include"
export LDFLAGS="-L$WTAPPS/lib -L$DJAPPS/lib"
cfg-djgpp \
$*
and ncurses:
#!/bin/sh
# Id: cfg-djgpp-ncurses,v 1.2 2010/05/22 17:28:57 tom Exp
cfg-djgpp \
--without-normal \
--with-debug \
--with-fallbacks=vt100,ansi,cygwin,linux,djgpp,djgpp203,djgpp204 \
--disable-database \
--enable-termcap \
--without-ada \
$*
In 2005, I noticed some newsgroup comments about cross-compiling from Linux to MinGW, e.g., in discussion of aeditor:
Newsgroups: comp.lang.ruby From: Tim Ferrell <Tim.Ferr...@mcgeecorp.com> Date: Sat, 5 Feb 2005 00:57:00 +0900 Subject: Re: aeditor-2.3 Reply | Reply to Author | Forward | Print | Individual Message | Show original | Report Abuse This link explains how to cross-compile for Windows from Linux... http://www.xraylith.wisc.edu/~khan/software/gnu-win32/mingw-cross-how. .. HTH, Tim
Later, in July 2008, Mark Robinson sent patches to allow vile and winvile to be built with cross-compiled MinGW. Since I was able to build both (with Visual Studio, and more capabilities), that was just an alternate way to build vi like emacs.
Ultimately what got my interest in cross-compiling for MinGW was Jürgen Pfeifer's patches to use MinGW for ncurses. He began developing those in September 2008, and I began integrating them into ncurses beginning in February 2009.
Earlier than this, I was certainly aware that ncurses did not work with MinGW (see mailing list comment from April 2006). But at that point I had thought that I might get ncurses to build and work in a Windows command window using Visual Studio C++. Because ncurses is built with shell scripts (and awk scripts) which create some of the source files, doing a Windows port in that way meant that I relied on generating those source-files on another system (the alternative, GnuWin32, simply could not handle those scripts).
Actually, native builds with MinGW were not as speedy or robust as I would have liked: Jürgen wanted to build 64-bit clients, which entailed adding a compiler from a different group, and modifying the MinGW configuration to work with it, as noted in README.MinGW:
Please also make sure that MSYS links to the correct directory containing your MinGW toolchain. For TDM this is usually C:\MinGW64. In your Windows CMD.EXE command shell go to the MSYS root directory (most probably C:\MSYS or C:\MSYS\1.0) and verify, that there is a junction point mingw that points to the MinGW toolchain directory. If not, delete the mingw directory and use the mklink command (or the linkd.exe utility on older Windows) to create the junction point.
Thereafter, if I wanted to reinstall MinGW, I would also have to reconstruct the configuration that supported the TDM compiler and get that to work again.
Although supporting Jürgen's changes for ncurses in MinGW got me involved with MinGW, it was vile (vi like emacs) which got me to experiment with cross-compilers which targeted MinGW:
Late in 2009 in discussing bug reports for vile, I realized that problems with my ports to 64-bit environments would be much simpler if I had more control over the machines (and the tools on which I was building my programs).
I found that 64-bit laptops were affordable, and got one.
At the beginning of 2010, I started configuring it. But unlike previous machines, this used GPT rather than MBR. One thing led to another, and I ended up reinstalling the machine.
In the final configuration, I had ten 64-bit virtual machines running in XenServer (which for me at that time involved no cost).
One of those was Windows 7. I set up a MinGW environment on that machine to use the TDM 64-bit compiler.
I also set up Visual Studio 2008 (the Express version of course: no cost). That was only 32-bits. On the same machine I also installed Cygwin, and later MSYS2.
Rick Sladkey, by the way, appeared to be following a similar path. He reported issues with the 64-bit compiler in Visual Studio.
I also had Fedora 12 and Mandriva. Both of those had packages for a cross-compiler to MinGW. I installed these in August 2010:
That is, 32-bit cross-compilers were available in 2010.
It took about two more years before I was using 64-bit cross-compilers for MinGW. For instance, I have logs for building vile from kFreeBSD and Debian/unstable during 2012.
Mentioning build-logs raises an issue: for several years I have collected logs from building my programs with the compilers available on each machine. Most of these build scripts use the same choices, so that adding the configuration to support cross-compiling vile would apply to other programs. I do not build everything all the time, though. I do these builds to check portability and correctness of whatever program I am currently working on. The ncurses build script is separate from the others because it builds and installs several configurations. I modified both scripts to attempt to build with MinGW in February 2012, to investigate the 64-bit cross compilers. My earlier builds were ad hoc, not as routine as they are now.
I began cross-compiling with simple programs, i.e., command-line tools which do not require curses. My logs show the initial dates for these:
A few of those were not done earlier, because the native Windows ports seemed more useful:
Building vile (vi-like-emacs) uses reflex; whether the DLLs needed for MinGW was not entirely clear.
Building conflict by cross-compiling can confuse the configure script because it depends on the type of pathnames.
I set those up using scripts to check for the availability of a cross-compiler, and if needed, a custom script for building the program. Because the packagers changed their configuration (and used different names on the different platforms), my script accounted for the different pathnames and filenames used for invoking the compiler.
Here is a script used for configuring with the 32-bit MinGW cross-compiler:
#!/bin/sh
# $Id: cfg-mingw,v 1.9 2018/01/04 00:19:10 tom Exp $
# configure to cross-compile using mingw32
BUILD_CC=${CC:-gcc}
unset CC
unset CXX
TARGET=`choose-mingw32 gcc pkg-config`
if test -n "$TARGET"
then
PREFIX=
test -d /usr/$TARGET && PREFIX="--prefix=/usr/$TARGET"
cfg-normal \
--with-build-cc=$BUILD_CC \
--host=$TARGET \
--target=$TARGET \
$PREFIX "$@" || exit 1
else
echo "? cannot find MinGW compiler in path"
exit 1
fi
The script for choosing the compiler (finding it) is longer:
#!/bin/sh
# $Id: choose-mingw,v 1.1 2018/01/04 00:06:57 tom Exp $
# vile:sw=4 ts=4
#------------------------------------------------------------------------------
# Choose a mingw "target" for 32/64 bits. This is especially for the Debian
# packages which provide old (gcc-mingw32) and new (gcc-mingw-w64)
# flavors, using symbolic links for more/less compatibility.
#
# Prefer the names which are not symlinked, but override that if more than
# one tool is specified.
[ $# = 0 ] && exec $0 gcc
save_IFS="$IFS"
IFS=':'
state=
target=
case "$0" in
*32)
prefixes=\
i686-w64-mingw32:\
i686-pc-mingw32:\
i686-mingw32msvc:\
i586-mingw32msvc:\
mingw32
;;
*)
prefixes=\
x86_64-w64-mingw32:\
amd64-mingw32msvc
;;
esac
for dir in $PATH
do
for tool in $prefixes
do
check=
for program in "$@"
do
path=$dir/$tool-$program
if [ ! -f "$path" ]
then
[ -n "$check" ] && check="miss"
break
fi
[ -z "$check" ] && check="file"
[ -h "$path" ] && check="link"
[ -t 2 ] || echo "$path" >&2
done
case "x$state" in
x)
if [ -n "$check" ]
then
state=$check
target=$tool
fi
;;
xlink)
case "x$check" in
xfile)
state=$check
target=$tool
;;
esac
;;
esac
if [ ! -t 2 ]
then
[ -n "$check" ] && printf '%s\t%s\t-> %s\n' "$state" "$check" "$target" >&2
fi
done
[ -n "$target" ] && break
done
IFS="$save_IFS"
echo $target
The larger programs (ncurses, vile, lynx), are a bigger problem in cross-compiling. Targeting the command-line of MinGW (using MSYS, an early fork from Cygwin according to the MSYS2 History) looked interesting, but how to make vile work with its screen was unclear. The MinGW installer does provide development packages for its command-line, but none of those appear relevant:
However, MinGW provides a copy of Vim, which uses the termcap DLL which is not available using MinGW's installer:
Seeing that, I lost interest. Others did not (see this relic).
When I have changes to make in the native MinGW environment, I use ded to navigate in a Cygwin window, and run vile from ded. In other environments and platforms, this workaround is unnecessary.
Cross-compiling vile to work in a Windows console window was more straightforward. Mark Robinson had introduced a reduced configuration convile (for the console driver ntconio.c), and minvile (for the graphical Windows driver ntwinio.c) in July 2008. I adapted that for cross-compiling in March 2012.
Because I have several programs, I wrote utility scripts for the MinGW and package builds, called from custom scripts. That helps when setting up a build script for a given program. The custom script for vile is short:
#!/bin/sh
# $Id: mingwbuild-vile,v 1.2 2013/10/08 00:14:40 tom Exp $
# build vile for win32 using mingw32 cross-compiler
. setup-mingwbuild
CONFIG_PROG=convile
CONFIG_OPTS="--with-external-filters --with-screen=dos"
perform_build
CONFIG_PROG=convile
CONFIG_OPTS="--with-builtin-filters --with-screen=dos"
perform_build
CONFIG_PROG=minvile
CONFIG_OPTS="--with-builtin-filters --with-screen=windows"
perform_build
The utility script which I began in February 2012 (for 32-bit MinGW) is longer:
#!/bin/sh
# $Id: setup-mingwbuild,v 1.19 2019/01/18 00:44:51 tom Exp $
# vile:ts=4 sw=4
# Build-support for MinGW-32 cross-compiling.
#
# wrappers set these variables:
# PROG - the generic program name
# CONFIG_PROG - program name which is built
# CONFIG_OPTS - configure options
BITS=32
MY_DESTDIR=BUILD-W$BITS
MY_LOGFILE=`partition`-mingw$BITS-run.log
MY_INSTALL=install
PKG_PREFIX=mingw$BITS-
PKG_QUERY="rpm -ql"
test -f /usr/bin/dpkg && PKG_QUERY="dpkg -L"
unset CDPATH
BUILD_DIR=`realpath .`
BUILD=$BUILD_DIR
failed() {
echo "? $*"
exit 1
}
. no-rcs
test_deb() {
TEST_DEB=`dpkg -l |grep $PKG_PREFIX'.*'$2 |head -n 1|awk '{print $2; }' | sed -e "s/$PKG_PREFIX//"`
eval $1="\$TEST_DEB"
test -n "$TEST_DEB" && echo "** found package $2 -> $TEST_DEB"
}
# copy files from package
from_pkg() {
echo "...from_pkg $*"
case $# in
0|1)
failed "from_pkg package destdir file1 [file2 ...]"
;;
*)
;;
esac
PACKAGE=$1
shift 1
TARGET=$1
mkdir -p $TARGET
shift 1
while test $# != 0
do
WANTED=`basename $1`
shift 1
SOURCE=`$PKG_QUERY $PKG_PREFIX$PACKAGE 2>/dev/null | egrep "/$WANTED\$" | head -n 1`
if test -n "$SOURCE"
then
copy -v $SOURCE $TARGET/
elif test -n "$BINARIES" && test -f $BINARIES/$WANTED
then
copy -v $BINARIES/$WANTED $TARGET/
else
failed "did not find $WANTED"
fi
done
}
# copy ".dll" file(s) from package
copy_dll() {
echo "...copy_dll $*"
case $# in
0|1)
failed "copy_dll package destdir [dlls]"
;;
2)
from_pkg $1 $2 `$PKG_QUERY $PKG_PREFIX$1 2>/dev/null | egrep '\.dll$'`
;;
*)
from_pkg $*
;;
esac
}
# When this file is source'd clean out our target directories.
clean_destdir() {
rm -rf $MY_DESTDIR
mkdir -p $MY_DESTDIR
test -d LOGS || mkdir LOGS
rm -f LOGS/$MY_LOGFILE
}
clean_srcdir() {
if test -f makefile || test -f Makefile
then
make distclean
fi
rm -f *.out
}
wrap_buildlog() {
cat <<EOF/
** `date`
** node: `hostname` (`partition`)
** user: `id`
EOF/
echo "** BUILD $CONFIG_PROG $CONFIG_OPTS"
case "x`partition`" in
*freebsd*)
NO_LOCAL=
;;
*)
NO_LOCAL=no-local
;;
esac
$NO_LOCAL cfg-mingw$BITS "$@" --prefix='' $CONFIG_OPTS || failed "configure"
if test -f makefile || test -f Makefile
then
make
make $MY_INSTALL DESTDIR=`realpath $MY_DESTDIR`
clean_srcdir
fi
}
# Build using mingwXX compiler installing into ./BUILD-WXX directory.
perform_build() {
clean_srcdir
rcs_save
wrap_buildlog "$@" 2>&1 | tee $MY_LOGFILE
if test -f $MY_LOGFILE
then
# split-up to avoid bogged-down buffering
sed -e "s,$BUILD/$MY_DESTDIR,DESTDIR,g" \
-e "s,$BUILD,SOURCE,g" < $MY_LOGFILE >>LOGS/$MY_LOGFILE
rm -f $MY_LOGFILE
fi
}
cfg-mingw --help 2>/dev/null >/dev/null || failed "no MinGW-$BITS compiler found"
clean_destdir
# Caller can setup more than one CONFIG_xxxx, calling perform_build multiple
# times.
Cross-compiling vile produces workable executables. However, the available tools for creating installers (mainly NullSoft) were too primitive to be useful. I began creating scripts in 2005 for Inno Setup, using those in the native Windows development environment for the executables that I provide for others (see vile's downloads).
I began cross-compiling ncurses in July 2012 (both 32-bit and 64-bit), using scripts to set the configure-options, e.g.,
#!/bin/sh
# $Id: mingwbuild-ncurses,v 1.1 2012/07/28 20:44:37 tom Exp $
# build for win32 using mingw32 cross-compiler
. setup-mingwbuild
CONFIG_PROG=ncurses32
CONFIG_OPTS="\
--with-shared \
--without-debug \
--with-trace \
--enable-term-driver \
--enable-sp-funcs \
--enable-widec \
--with-fallbacks=unknown,rxvt \
--with-tparm-arg=intptr_t \
--without-ada"
perform_build
The --with-shared option tells the configure script to produce DLLs for this environment. The native MinGW tools did not support the options which I had found for creating DLLs. The cross-development tools did this.
Later (in October 2013), I created Debian package scripts for testing the DLLs when building my other programs. I provide the resulting DLLs in zip-files, for other developers. End users generally use packages built by others, as I mentioned here:
ncurses 6.0 was first packaged for MinGW in March 2016 by Erwin Waterlander:
Later, it was added to the MinGW installer's inventory. This is in the contributed section, like pdcurses:[Mingw-users] [ANNOUNCEMENT] New: msys-ncurses 6.0 From: Erwin Waterlander <water...@xs...> - 2016-03-16 19:59:30 Hi, A new package is available: msys-ncurses 6.0. This package is not installable via mingw-get, but needs manual installation. You can download the files from: https://sourceforge.net/projects/mingw/files/MSYS/Contributed/ncurses/ncurses-6.0-2/ Only narrow text is supported (no wide text (Unicode)). This library can be used for msys applications in the mintty terminal emulator. msys-ncurses will not work in Windows consoles like Windows Console, Console(2), ConsoleZ, or ConEmu. To run msys in mintty type: msys.bat -mintty MSYS or for a mingw32 environment: msys.bat -mintty MINGW32 regards, -- Erwin Waterlander http://waterlan.home.xs4all.nl/
The DLLs which I build run in Windows consoles, as outlined in Juergen's README.MinGW. However, Keith Marshall pointed out (in August 2020) that these use different options than he uses for building MinGW:
I'm looking for ncurses, to interoperate with a MinGW GCC-9.2.0 compiler tool chain. I first tried https://invisible-mirror.net/archives/ncurses/win32/mingw32.zip, but that is broken, because its libncursesw6.dll depends on libgcc_s_sjlj-1.dll $ mingw32-ldd ~/VirtualBox/share/libncursesw6.dll /home/keith/VirtualBox/share/libncursesw6.dll +- KERNEL32.dll +- msvcrt.dll +- USER32.dll +- libgcc_s_sjlj-1.dll FYI, no legitimately published 32-bit MinGW compiler will introduce any such dependency; any which does is illegitimately published, insofar as it masquerades as a MinGW product, infringing the MinGW trademark[1]. Since this clearly isn't compatible with a real MinGW 32-bit compiler, (which requires libgcc_s_dw2-1.dll), I downloaded the source from https://invisible-mirror.net/archives/ncurses/ncurses-6.2.tar.gz, ...
That is a result of cross-compiling rather than native builds. On my Debian machine, for instance, I have only these:
$ locate libgcc |grep dll /usr/lib/gcc/i686-w64-mingw32/8.3-posix/libgcc_s_sjlj-1.dll /usr/lib/gcc/i686-w64-mingw32/8.3-win32/libgcc_s_sjlj-1.dll /usr/lib/gcc/x86_64-w64-mingw32/8.3-posix/libgcc_s_seh-1.dll /usr/lib/gcc/x86_64-w64-mingw32/8.3-win32/libgcc_s_seh-1.dll
However, I could build using Fedora:
$ locate libgcc | grep dll /usr/i686-w64-mingw32/sys-root/mingw/bin/libgcc_s_dw2-1.dll /usr/x86_64-w64-mingw32/sys-root/mingw/bin/libgcc_s_seh-1.dll
but generally use Debian because they provide the most bug reports.
Cross-compiling lynx is more challenging, because (unlike vile) it does not provide a Windows console display driver. It uses curses or (with some restrictions) slang.
To cross-compile an application which uses external libraries, you need the libraries (at least the import libraries) available while cross-compiling.
I began cross-compiling lynx in 2012, first trying Ubuntu 11.10 (February), then Debian 6 and Fedora 16 (August). Like my other programs, I use a build-all-XXX script for testing updates. I added this chunk to the script for MinGW64:
if ( cfg-mingw --version 2>/dev/null >/dev/null )
then
mingwbuild-$PROG && run-makensis */*.nsi || rmdir BUILD-W32
test "$HAVE_RPM" = yes && rpmbuild-mingw-$PROG
test "$HAVE_DEB" = yes && debbuild-mingw-$PROG
else
echo "** MINGW-32 not found"
fi
if ( cfg-mingw64 --version 2>/dev/null >/dev/null )
then
mingw64build-$PROG && run-makensis */*.nsi || rmdir BUILD-W64
test "$HAVE_RPM" = yes && rpmbuild-mingw64-$PROG
test "$HAVE_DEB" = yes && debbuild-mingw64-$PROG
else
echo "** MINGW-64 not found"
fi
Debian provides the tools, but very few libraries. Checking with aptitude, I see 6. Only libz-mingw-w64-dev is useful for lynx. If I want anything more in Debian, I will have to build it myself.
Fedora provides a package for PDCurses which can be used in cross-compiling to MinGW. It packages neither ncurses nor slang for cross-compiling. Both can run in a Windows console.
For the record, the packager's statement that “ncurses is not available for MinGW / Windows” dates from January 2009 — about a month before I began integrating Juergen's changes to port ncurses to MinGW. The port was stable sometime before I released ncurses 5.8 in 2011.
Unlike the other programs, Lynx supports several configurations that depend upon external libraries.
My native builds on Windows use more than one environment, with corresponding build-trees.
The last three are under the home-directory for my development account, in the corresponding environment. As in my Linux and Unix machines, each version of a program has its own directory. I use RCS on the development file-server for source control. I update these Windows environments using rsync and WinSCP.
I do test-compiles with all of the versions of Visual Studio which I am able to install. In 2019-2020, I revamped my development system for building Lynx. I used batch-files from WinVile to run “command-line” builds for each of Lynx's dependencies and collected those into a directory tree, which is a local Git repository.
Here is a screenshot of that directory tree:
The Visual Studio build-tree is more static; directory names do not change often. Here is a screenshot of that directory tree:
I maintain the patches needed to build those libraries via WinSCP. In particular, slang requires patching because its developer has not provided a suitable source for more than ten years.
Besides the compression libraries (bzip2, gzip, zlib), Lynx uses OpenSSL libraries which I use as mentioned on my Lynx development page. I decided to not build my own gettext and iconv libraries. Some people do. For example:
Look closer, and notice that those have to be patched and maintained.
Most of the programs which I cross-compile to MinGW have few dependencies. “MinVile” and ncurses use nothing more than the DLLs which are either standard, or are part of the cross-compiler toolchain. Oddly enough, those toolchains do not include a tool to show dependencies, such as ldd.
For Windows development, these are useful, with a few caveats:Caveats:
These tools give only the static (compile/link time) dependencies.
Later versions of Windows introduce features which make the tool obsolete.
Still, there was no analogous cross-compiling tool in 2015, according to the answer given for Finding DLLs required of a Win exe on Linux (cross-compiled with mingw)?
A year later (October 2016), Keith Marshall wrote a script (mingw32-ldd.sh) which uses the native- or cross-compiler's objdump to analyze an executable (e.g., .exe or .dll) and show the DLLs which it references. The script works in both MinGW32 (native) and MinGW (cross-compiled) environments. For the latter, the user is expected to rename the script to use the same prefix as the other cross-compiling tools.
However, Debian and Fedora use different prefixes (and those have changed since 2012). I use scripts to choose the appropriate prefix for the MinGW32 and MinGW64 toolchain from lists which cover different platforms rather than make a lot of copies of Keith's script:
#!/bin/bash
# -----------------------------------------------------------------------------
# Copyright 2023 by Thomas E. Dickey
...
# -----------------------------------------------------------------------------
# This is a wrapper for Keith Marshall's "mingw32-ldd.sh" script, which along
# with my choose-mingw32 and choose-mingw64 scripts, automatically selects
# the proper target based on the output from "file".
mytemp=$(mktemp -d)
trap 'rm -rf "$mytemp"; exit 1' INT QUIT TERM HUP
trap 'rm -rf "$mytemp"; exit 0' EXIT
PATH=$mytemp:$PATH
export PATH
for name in "$@"
do
[ -f "$name" ] || continue
case $(file "$name") in
*PE32*386*)
target=$(choose-mingw32 2>/dev/null)
;;
*PE32*x86-64*)
target=$(choose-mingw64 2>/dev/null)
;;
*)
continue
;;
esac
script="$mytemp"/"$target"-ldd.sh
if [ ! -f "$script" ]
then
echo "#!/bin/bash" > "$script"
echo ". mingw32-ldd.sh" >> "$script"
chmod +x "$script"
fi
$script "$name"
done
# vile:ts=4 sw=4
Here are listings for ncurses (built on Debian 10), with 32-bit:
$ mingw-ldd libncursesw6.dll libncursesw6.dll +- KERNEL32.dll +- msvcrt.dll +- USER32.dll +- libgcc_s_sjlj-1.dll
and 64-bit:
$ mingw-ldd libncursesw6.dll libncursesw6.dll +- KERNEL32.dll +- msvcrt.dll +- USER32.dll
With Debian 11, the 32-bit dependencies changed, as noted in debian-devel-changes:
gcc-mingw-w64 (24) unstable; urgency=medium
* This release of gcc-mingw-w64 changes the 32-bit exception handling
mechanism from SJLJ to Dwarf2, to match MSYS2, Fedora, and other
toolchains, and to allow the Rust toolchain to provide 32-bit
cross-compilers.
This will require rebuilding all artifacts built with previous
releases of the toolchain. Shared binaries will need to be shipped
with the dw2 DLL instead of the sjlj DLL.
-- Stephen Kitt <skitt@debian.org> Sat, 21 Nov 2020 14:55:37 +0100
i.e.,
$ mingw-ldd libncursesw6.dll libncursesw6.dll +- KERNEL32.dll +- msvcrt.dll +- USER32.dll +- libgcc_s_dw2-1.dll
but the 64-bit dependencies did not.
Having mingw-ldd can simplify the problem of providing the required DLLs when cross-compiling lynx. The setup-mingwbuild script has functions for copying DLLs from development packages. I used that in the mingwbuild-lynx script:
#!/bin/sh
# $Id: mingwbuild-lynx,v 1.14 2023/01/24 00:33:34 tom Exp $
# vile:ts=4 sw=4
# build for win32 using mingw32 cross-compiler
copy_binaries() {
BINARIES=/usr/build/lynx/package-2.8.8dev.16-iss
from_pkg bzip2 $MY_DESTDIR/bin/ bzip2.exe
copy -v $BINARIES/gzip.exe $MY_DESTDIR/bin/
copy_dll win-iconv $MY_DESTDIR/bin/
copy_dll gettext $MY_DESTDIR/bin/
# /usr/i686-w64-mingw32/sys-root/mingw/bin/libgcc_s_sjlj-1.dll
copy_dll gcc $MY_DESTDIR/bin/
case "$CONFIG_OPTS" in
*disable-idn*)
;;
*)
copy_dll libidn2 $MY_DESTDIR/bin/
copy_dll libidn $MY_DESTDIR/bin/
;;
esac
case "$CONFIG_OPTS" in
*with-brotli*)
copy_dll brotli $MY_DESTDIR/bin/
;;
esac
case "$CONFIG_OPTS" in
*with-bzlib*)
copy_dll $PKG_BZLIB $MY_DESTDIR/bin/
;;
esac
case "$CONFIG_OPTS" in
*with-screen=pdcurses*)
if test -f /usr/bin/dpkg
then
copy_dll pdcurses3 $MY_DESTDIR/bin/
else
copy_dll pdcurses $MY_DESTDIR/bin/
fi
;;
esac
case "$CONFIG_OPTS" in
*with-ssl*)
copy_dll openssl $MY_DESTDIR/bin/
;;
esac
case "$CONFIG_OPTS" in
*with-zlib*)
copy_dll $PKG_ZLIB $MY_DESTDIR/bin/
;;
esac
}
. setup-mingwbuild
PKG_BZLIB=bzip2
PKG_ZLIB=zlib
CONFIG_PROG=lynx
CONFIG_OPTS=
case `partition` in
*fedora*)
CONFIG_OPTS="$CONFIG_OPTS --with-ssl"
CONFIG_OPTS="$CONFIG_OPTS --with-brotli"
CONFIG_OPTS="$CONFIG_OPTS --with-bzlib"
CONFIG_OPTS="$CONFIG_OPTS --with-zlib"
CONFIG_OPTS="$CONFIG_OPTS --enable-gzip-help"
CONFIG_OPTS="$CONFIG_OPTS --enable-htmlized-cfg"
CONFIG_OPTS="$CONFIG_OPTS --with-screen=pdcurses"
;;
*)
test_deb PKG_BZLIB libbz2
if test -n "$PKG_BZLIB"
then
CONFIG_OPTS="$CONFIG_OPTS --with-bzlib"
fi
test_deb PKG_ZLIB libz
if test -n "$PKG_ZLIB"
then
CONFIG_OPTS="$CONFIG_OPTS --with-zlib"
CONFIG_OPTS="$CONFIG_OPTS --enable-gzip-help"
CONFIG_OPTS="$CONFIG_OPTS --enable-htmlized-cfg"
fi
CONFIG_OPTS="$CONFIG_OPTS --with-screen=pdcurses"
;;
esac
MY_INSTALL=install-full
perform_build
copy_binaries 2>&1 | tee -a LOGS/$MY_LOGFILE
That case-statement is cumbersome.
Here is a listing for (32-bit) Lynx created on Fedora:
$ mingw-ldd lynx.exe lynx.exe +- ADVAPI32.dll +- libbz2-1.dll | +- KERNEL32.dll | +- msvcrt.dll +- libcrypto-3.dll | +- ADVAPI32.dll | +- KERNEL32.dll | +- msvcrt.dll | +- api-ms-win-core-path-l1-1-0.dll | +- USER32.dll | +- WS2_32.dll | +- zlib1.dll | | +- KERNEL32.dll | | +- msvcrt.dll | +- libssp-0.dll +- iconv.dll | +- KERNEL32.dll | +- msvcrt.dll | +- libssp-0.dll +- libidn2-0.dll | +- libgcc_s_dw2-1.dll | +- KERNEL32.dll | +- msvcrt.dll | +- libssp-0.dll +- KERNEL32.dll +- msvcrt.dll +- pdcurses.dll | +- ADVAPI32.dll | +- libgcc_s_dw2-1.dll | +- KERNEL32.dll | +- msvcrt.dll | +- USER32.dll +- libssl-3.dll | +- libcrypto-3.dll | | +- ADVAPI32.dll | | +- KERNEL32.dll | | +- msvcrt.dll | | +- api-ms-win-core-path-l1-1-0.dll | | +- USER32.dll | | +- WS2_32.dll | | +- zlib1.dll | | | +- KERNEL32.dll | | | +- msvcrt.dll | | +- libssp-0.dll | +- KERNEL32.dll | +- msvcrt.dll | +- libssp-0.dll +- USER32.dll +- WS2_32.dll +- zlib1.dll | +- KERNEL32.dll | +- msvcrt.dll +- libssp-0.dll
Using the information from mingw-ldd, here is an example of the direct dependencies for Lynx with Fedora:
total 9436 -rwxr-xr-x. 1 tom users 36595 Jan 18 19:00 iconv.dll -rwxr-xr-x. 1 tom users 112718 Jan 18 19:00 libbz2-1.dll -rwxr-xr-x. 1 tom users 4132118 Jan 18 19:00 libcrypto-3.dll -rwxr-xr-x. 1 tom users 410601 Jan 18 19:00 libidn2-0.dll -rwxr-xr-x. 1 tom users 738809 Jan 18 19:00 libssl-3.dll -rwxr-xr-x. 1 tom users 205137 Jan 18 19:00 pdcurses.dll -rwxr-xr-x. 1 tom users 134385 Jan 18 19:00 zlib1.dll -rwxr-xr-x. 1 tom users 78724 Jan 18 19:00 bzip2.exe -r--r--r--. 1 tom users 68096 Oct 15 2007 gzip.exe -rwxr-xr-x. 2 tom users 1861382 Feb 20 10:27 i686-w64-mingw32-lynx.exe -rwxr-xr-x. 2 tom users 1861382 Feb 20 10:27 lynx.exe
However, some of those have further dependencies, and can be overlooked if the list is maintained manually. Here is a listing showing all of the dependencies:
ADVAPI32.dll ? KERNEL32.dll ? USER32.dll ? WS2_32.dll ? api-ms-win-core-path-l1-1-0.dll ? iconv.dll libbz2-1.dll libcrypto-3.dll /usr/i686-w64-mingw32/sys-root/mingw/bin/libgcc_s_dw2-1.dll libidn2-0.dll libssl-3.dll /usr/i686-w64-mingw32/sys-root/mingw/bin/libssp-0.dll msvcrt.dll ? pdcurses.dll zlib1.dll bzip2.exe gzip.exe lynx.exe
The listing shows two types of extra dependencies:
The lines with “?” are Windows system DLLs which are not wanted in a package (and are not provided in cross-compiler development packages).
The lines with an absolute pathname show the missing external DLLs which should be added to a package. In a Windows package, all of those files would be in the same (non-system) directory, so that they do not conflict with other programs that may supply a different version of the DLLs.
The missing DLLs are found in the cross compiler support package. They may be present on the target system, but providing them in a package ensures that they are present when needed in a minimal cross-compiled package.