slashpackage-foreign

I install non-slashpackage packages in subdirectories of /package/misc/spf, which I have registered for this purpose in the slashpackage registry. This system, called spf for short, is discussed on the spf list. Of course, there is some overlap with the slashpackage list as well.

Although all packages are installed under /package/misc/spf, they need not all be stored on the same filesystem. See this page for details.

spftools

While many of the ideas here remain intact, the code is being rewritten. spftools is a work in progress, so if you've been using sp-foreign, you probably shouldn't stop right now, but eventually spftools will supersede sp-foreign. sp-foreign will only be updated for some new package versions now; all new development focuses on spftools.

Caveats

Warning: Installing packages in these non-default locations solves some problems (easy, reliable upgrading and uninstallation; multiple simultaneously-installed versions for testing; etc.) and creates others (other software expecting to find these packages in their respective default locations will fail, etc.). Read this page and this page and make sure you understand the burden you are bringing on yourself before using spf. Consider setting up symlinks in the default locations pointing to the package's files via the current-version symlink. sp-foreign does this for you in some cases. Do not ask package authors to automatically look for dependencies in spf locations; making this work is our problem, not theirs.

Also, a general warning for building a system completely from source: you might not be binary-compatible with, say, RedHat, so you would not be able to install RedHat's binary RPMs. You'll be limited to software that you can compile yourself.

Scope

This project is intended to provide a more-or-less complete OS distribution. (Though you should also be able to use it to install individual packages on traditional systems; e.g., I've used it to install openssh and its dependencies on Solaris.) spf happens to not repackage the original sources; it consists only of the additional scripting needed to install packages in this style. This project will almost certainly never be as full-featured as other OSes: ideally, the packages' authors would convert their packages to slashpackage before we do, and packages that are distributed in slashpackage style are not included in this project. This does not mean that they are not considered useful, but this project is only concerned with the "leftover" pieces that are needed to make a working system.

Feel free to ask authors to distribute their packages in slashpackage style themselves, but be warned that you will likely meet with resistance. Possibly the best approach to bring slashpackage to the world would be to add functionality to the GNU autotools to make it easy to produce both slashpackage and non-slashpackage tarballs. Then other packages will have a much easier time converting, and can continue to distribute traditional-style tarballs as well.

Eventually, I will clean up my boot loader configuration, boot/shutdown scripts, etc., and publish them either as part of this project or separately. If you're planning to install a complete spf system, you'll have to fill in the gaps yourself for now. Some of the missing pieces are available here and here.

Aside from a few patches below, I do not provide updates for security-related or other problems in particular packages. The packages' own maintainers are responsible for their quality. You can read bugtraq yourself. If someone else wants to take on the task of releasing patched sources in a more timely fashion than the packages' own maintainers do, then that is a project that would cooperate well with spf (and with other distributions; factoring is good design). Of course, the Right Way is to get the packages' own maintainers to release updates more promptly.

I do not provide binaries. If someone else wants to take on that task, then that is a project that would cooperate well with spf. Some binaries are available at http://spf.relaxism.com/.

My own systems are completely spf. If you'd like to set up your systems the same way, but you need packages that are not included in spf, you can request that they be included. I don't promise that I will include every request, but I will try to make spf a generally useful system.

Configuration

These tools use environment variables to configure certain parameters.

• The variable $SPF_PREFIX specifies the directory that contains the package installation directories; it defaults to $SP_ROOT/package/misc/spf in both the gcc wrapper and sp-foreign. $SPF_PREFIX may be dropped in the future; $SP_ROOT is the recommended way to configure the path.
• The variable $SPF_CONF specifies the path of a package-specific directory where symlinks to dependencies are created. It is used by the gcc wrapper. sp-foreign sets it to $SPF_PREFIX/<package>-<version>/spf. If you want to link to your spf installation of glibc, you must set this variable yourself when installing non-spf packages; for example:

      cd /package/admin/daemontools-0.76
      SPF_CONF="$PWD/spf" package/compile

  sp-install does this for you automatically.

gcc wrapper

If you install your C library or gcc with spf, you'll need a wrapper for gcc to find libc and libgcc.so when compiling other packages. This will be needed in order to build any package including C code, not just spf packages (for libc, anyway; not all packages need libgcc.so). If no libc is found in $SPF_PREFIX, the wrapper will not modify the gcc invocation to find different libc files, and gcc will therefore use the libc files found in the traditional places: /lib, /usr/lib, and /usr/include. For now, you can install the wrapper manually as /package/misc/spf/spf/wrapper/gcc and make a symlink to it in /command. It should be found first in $PATH by the names gcc, g++, c++, cc, cpp, g77, gcj, and the corresponding target-specific names (i686-pc-linux-gnu-*, etc.). I don't know whether any wrapping is needed for other programs. The wrapper notices certain environment variables, in addition to the two described above:

• SPF_LIBC: set this to "glibc" (the default) or "dietlibc" (not well tested) to specify which libc to use. (I do not recommend using "diet gcc" directly when gcc is this wrapper; use "env SPF_LIBC=dietlibc gcc" instead.)
• SPF_GCC: set this to the path of a gcc installation to use one other than /package/misc/spf/gcc.
• SPF_GCC_GCCLIB: set this to any value to indicate that gcc's lib/ directory should be included in the runtime search path for the programs being compiled. This is needed if the programs use libgcc.so, libstdc++.so, etc.
• SPF_GCC_DEBUG: set this to any value to make the wrapper log its activity to descriptor 3. This will show the changes made by the wrapper to the command line arguments and environment variables.
• SPF_GCC_CFLAGS: set this to any arguments you want to add to the end of the gcc command line. This is intended for user customization; programs should not mangle it. It is used for g77 and gcj invocations as well as gcc and g++, so it is not the best place for C-specific -I flags, etc. (But most packages don't use g77 or gcj.)

sp-foreign

The sp-foreign program (compressed) builds and installs particular packages. The packages' sources and a few patches are mirrored here. For now, you can install it manually as /package/misc/spf/spf/command/sp-foreign and make a symlink to it in /command.

Building a complete spf system

You can use sp-foreign to install just individual packages, or to install/replace a complete system. To install a complete spf system over an existing system, start by installing binutils. (You may have to install m4, bison, and flex first if the system doesn't already have them.) Do not use the gcc wrapper yet, or if you do, remember to set $SPF_GCC to the path to your existing gcc installation prefix. As you install each package, set the current-version symlink (e.g., with sp-version), make symlinks to its commands in /command (e.g., with sp-links), and make sure /command occurs before other systemwide directories in your $PATH. Next, install gcc. Once this gcc is installed, you can arrange for the wrapper to be found first in $PATH, and you can skip setting $SPF_GCC when installing further packages, unless you specifically want to use a different gcc installation. Next install linux-libc-headers and tzdata, and then glibc. Now the gcc wrapper script will find this glibc, so install new copies of the packages you've installed so far other than glibc (setting $SPF_BUILD to create new installations instead of overwriting the old installations), to have them linked to your new glibc, starting with gcc and then binutils. Continue on with whichever packages you want.

Installing a new spf system

Eventually, I might provide more automated first-time installation tools, but for now, to install spf on a new system, it helps to install a different system first, just as with Linux from Scratch. (Another approach would be to boot the new system from a floppy and copy binaries from an existing spf system.)

If you want the root filesystem to be, say, ReiserFS, but the first system (I'll call it the "bootstrap" system) installer doesn't allow you to create and boot from that kind of filesystem, I suggest putting the bootstrap system on a different filesystem from the one that will later be the spf root filesystem. For example, I just set up a new system doing this. I planned to have four filesystems: one for root, one for packages, one for logs/queues/etc., and one for home directories. I organize my filesystems using /fs. First I installed Red Hat; I set up the partitions the way I want them on the spf system, but I installed Red Hat on the home filesystem instead of root. Then, on the home filesystem, I created a /fs hierarchy containing the following entries: spf-root, pkgs, and logs. I set up symlinks in the Red Hat root directory pointing to directories on the other filesystems: command and package pointing into the pkgs filesystem, and slash pointing into the spf-root filesystem. Then I installed packages normally. When I had enough installed in the spf system, I rebooted into it, using the spf root as the root filesystem. Then I remade the home filesystem as ReiserFS, overwriting the Red Hat system.

Versions of major packages

The versions you choose for basic development tools are important. Major upgrades often do not keep binary compatibility. glibc should be built by approximately the same versions of gcc and binutils that you will use for everything else; that is why you should install gcc and binutils first. But gcc assumes to some degree (via fixincludes, if nothing else) that it will build software using the same libc that it was built with; that is why gcc should be reinstalled first after glibc. And presumably, you will delete your old glibc (and everything else) after you've installed a complete spf system; that is why you should reinstall everything after installing glibc.

Upgrading major packages

It is possible to upgrade a whole system to new major versions of the basic development tools without any interruption of service, by manipulating the per-package spf/* symlinks in the old packages to point to specific versions of all dependencies. (sp-freeze can help with that.) Then the new packages can be installed, starting with binutils/gcc/glibc/gcc/binutils again. The current-version symlinks will be updated as each new package is installed, but the old copies of other packages will still use the old versions of their dependencies. After all packages have been reinstalled, the old copies can be removed.

Remaining traditional files

After you install a complete system in /package, you can remove your old system components. My systems have just a few files left in the traditional locations. /bin and /sbin exist for compatibility as symlinks to /command. /usr is a symlink to /. /local (normally accessed via the /usr symlink) contains bin and sbin as symlinks to /command. The following files remain as in a traditional GNU/Linux system: /dev, /etc/fstab (or not), /etc/group, /etc/leapsecs.dat, /etc/modules.conf for kernel 2.4 systems (which includes /package/misc/spf/linux/modules.conf), /etc/mtab (a symlink to /proc/mounts), /etc/nsswitch.conf, /etc/passwd, /etc/protocols (a symlink to a file from iana-etc), /etc/resolv.conf, /etc/services (a symlink to a file from iana-etc), /etc/shadow, /etc/shells, /home, /proc (sort of), /tmp, /var/log/wtmp, and /var/run/utmp.

Using sp-foreign

Example use: sp-foreign gcc 2.95.3 --enable-languages=c,c++

The first two arguments are the package name and version, used to construct the installation prefix and find the source directory. The remaining arguments, if any, are passed to the package's build system in some way specific to the package. The interpretation of those remaining arguments may change in the future on a per-package basis; this is not entirely stable yet. You may want to save a copy of this page for reference along with the script itself. $CPPFLAGS, $CFLAGS, and $LDFLAGS are passed to each package's build system, when possible, along with the flags sp-foreign itself provides. The intent is that the flags you specify should override those provided by sp-foreign.

sp-foreign will look for the package sources in the directory that the package's tarball unpacks into (typically "./<package>-<version>") and install the package in $SPF_PREFIX/<package>-<version>$SPF_BUILD. $SPF_BUILD must be used to differentiate between different installations of the same version, if you have more than one such installation at the same time (e.g., with different compile-time configurations); you cannot generally reinstall over an existing installation. The source directory is expected to be clean: possibly patched, but not yet built. The installation prefix is expected to be empty or nonexistent.

Although $SPF_PREFIX will probably be owned by root, you can install most packages as a non-root user by creating the package's directory as root and changing its ownership to the non-root user before running sp-foreign. Afterwards, you can change the ownership of the whole package back to root. Beware that programs which are normally setuid root may be installed with or without the setuid bit if installed as a non-root user.

sp-foreign notices certain environment variables. If $SPF_BARE is set, then some stripping-down configuration options are used for certain packages, noted below. If $SPF_NO_NLS is set, then NLS support is disabled in packages that support it. If $SPF_COMPAT is set to "n", then sp-foreign will not create compatibility symlinks such as /var/qmail or /usr/local/bglibs.

When package X-1.0 uses absolute compile-time paths to find files belonging to package Y, a symlink is created as $SPF_PREFIX/X-1.0/spf/Y pointing to $SPF_PREFIX/Y (unless such a symlink already exists), and X is configured to use Y via the symlink in its spf directory. You can change this symlink even after installation if you want to make X use a different installation of Y. If you want to use an installation of Y somewhere other than $SPF_PREFIX, you can create a symlink in $SPF_PREFIX/X-1.0/conf/ pointing to Y's installed prefix, before running sp-foreign, so sp-foreign can find it when installing X. (This is an exception to the expectation that the installation prefix directory should be initially empty.)

If you set $SP_IGNORE_TEST_FAIL=y and the package's tests fail, then sp-foreign will print a warning but exit successfully. (Normally, sp-foreign would exit nonzero to indicate failure.) This is useful when scripting a large number of package installations, and you want to abort for installation errors but not test failures.

This table shows the package versions that sp-foreign was developed for. (If you know of newer versions, please let me know. I subscribe to announcement lists when they exist, but sometimes they don't.) Hopefully, future versions won't break sp-foreign, but they might. Before using this script, you should probably review the configuration options it uses, in case you want to use different options. There may also be preliminary/untested support for some packages not listed here. Others have also written build scripts for sp-foreign:

• Alan Grosskurth
• Daniel Kruszyna
• Thomas Schwinge

To install packages whose build scripts are not included directly in sp-foreign, set $SP_FOREIGN_DIR to a directory containing the extra scripts.

include /package/misc/spf/linux/modules.conf