Table of Contents
The easiest way to write programs to work under LRP is to use Debian 2.1 (Slink). This is because of several factors:
Debian 2.1 was the original base for LRP, and thus was modeled after it, and uses programs from it.
Debian 2.1 uses glibc 2.0.7 — which LRP needs — and not glibc 2.1.x.
Most LRP developers use Debian 2.1.
However, other systems and Linux distributions can be used. The only requirement is that compiled programs be linked against the GNU 2.0 C libraries, not the more current 2.1 version. This requires special attention and configuration for current Linux distributions.
The gcc How-to is most helpful. It is available at http://tldp.org/HOWTO/GCC-HOWTO/index.html.
Common problems include the following:
The standard development environment (Debian 2.1 or Red Hat 5.2) typically uses Linux 2.0; some programs require headers included in Linux 2.2 or better. This can be rectified by upgrading the kernel — usually not a problem.
LRP contains glibc 2.0.7; most modern programs are now compiled against glibc 2.1 or glibc 2.2 — much, much bigger libraries. This is a very big problem.
Sometimes a program's source relies on definitions found in glibc 2.1 or 2.2 header files but missing in glibc 2.0 header files, or located elsewhere in the glibc 2.0 header files, or defined differently. The worst possible result of this is that the sources cannot be compiled under glibc 2.0. This is the case for the new bridge utilities and others. Fortunately, this drastic result does not happen often.
Many networking programs use libpcap (the Packet Capture Library) and look for pcap.h in /usr/include; change the include files to look for pcap.h in /usr/include/pcap — make sure, of course, that you really do have libpcap first.
LRP uses glibc 2.0.x[1], not glibc 2.1.x or glibc 2.2.x. This is not immediately obvious, but shows up quickly if any programs linked against glibc 2.1 are run under LRP. The results are invariably a segmentation fault. A program must be compiled to use glibc 2.0, not glibc 2.1, if it is to run under current versions of LRP and its derivatives.
Be sure that you get the most current version of the GNU C libraries, whatever version you use; there have been security patches applied to nearly every version. Security problems in glibc affect your entire system; thus, it is important to upgrade as necessary.
Being able to work with static libraries is important, since depending on the library it may or may not be included in the LRP system. For example, the GNU C libraries are included, but the Linux pthreads library is not. Thus, either the Linux pthreads library will have to be statically linked with an executable that requires it or the pthreads library will have to be added to LRP. A static library is one that is included with the resulting executable, thus reducing the need for external libraries.
Static libraries have tradeoffs, however: the size of the executable increases accordingly, since the library is now included in the executable. Another tradeoff is that a statically-linked library may be included on a system multiple times if multiple executables contain it. If a dynamic library is used, it exists only once — but a non-traditional library (such as libpcap.so or libpthreads.so) is then required by the binary to execute, and the error messages resulting from a missing library is not immediately understandable by a non-technical user. The gcc How-To has a valuable description of the linking process, including working with static and dynamic libraries.
To modify a make file to use only specific static libraries (and not create a single statically linked executable), change:
LIBS=-lncurses
to
LIBS=-Wl,-Bstatic -lncurses -Wl,-Bdynamic
Note that that is an ell, not a one, and that the second hyphen (or dash) is required.
To create dynamic libraries where only static libraries exist, use one of the following methods:
Use the linker[2]
ld --whole-archive -shared -Wl,-soname,libfoo.so.1 -o \ libfoo.so.1.o libfoo.a
Use the C compiler
gcc -shared -Wl,-soname,libfoo.so.1 -o libfoo.so.1.0 *.o
The former is probably prefered, since it actually uses a static library as its input, not the component object files.
The glibc2 How-To is a valuable information resource; it is at http://tldp.org/HOWTO/Glibc2-HOWTO.html
The gcc How-To is also valuable; it is at http://tldp.org/HOWTO/GCC-HOWTO/index.html
The simple answer is size. Glibc 2.1 increased in size dramatically; apparently this is due to the library introducing support for 64-bit environments like the new upcoming Intel chips.
However, there is in development an Oxygen variant with glibc 2.1.3. There was a significant tradeoff in terms of disk space (150k or more).
The most recent version of glibc 2.1 is 2.1.3.
The simple answer here is nobody's looked at it yet. It will depend on the size of the current libraries and their stability.
The most recent version of glibc 2.2 is 2.2.3.
The most recent version of Oxygen permits easy updating of glibc by replacing libc.lrp with whichever version is desired.
The following are the standard library files included in LRP under /lib. All are part of glibc v2.0.7, except for libuuid, libss, and libncurses. There are no files included under /usr/lib in the standard LRP.
Placing new libraries into LRP is a fairly simple matter for most. Simply put the executable in one of these places:
The preferred place is probably most often /usr/lib, since the system libraries are in lib. /Note that in Oxygen, /usr/lib is backed up with usr.lrp, and in all versions, /usr/local/lib is backed up with local.lrp. This means that libraries in these directories are not available during boot the way /lib is. Thus, any executable that requires a library not in /lib must not (and indeed, cannot) be used during the boot process. Any binary to be used during booting must rely only on dynamic libraries located in /bin.
Upgrading libraries in a current running system (especially glibc libraries) is asking for trouble. It is better to unpack the root.lrp package into a working directory on a development system, then replace the binaries and put root.lrp back on disk. This development system can be a LRP system but the replacement takes place in the tar archive, not the running system. Unpacking the archive would go something like this:
# cd /tmp # mkdir work # cd work # gunzip -c - < /mnt/root.lrp | tar xvf -
Now the libraries can be replaced. Make sure the links are set properly, and that the version numbers are set appropriately. For example, libcrypt may consist of the following files:
libcrypt-2.0.7.so The real file libcrypt.so.1 A symbolic link to libcrypt-2.0.7.so
Once the files are in place, do:
# tar cvf - * | gzip -9 -c - > ../root.lrp
Now you have a new root.lrp in /tmp to put on disk as desired.
Compiling source files for LRP requires making gcc use version 2.0.x of the C libraries. If the system comes with this version (as Debian 2.1 and Red Hat 5.2 do) then nothing more is required. More recent Linux distributions use version 2.1.x.
To force gcc to compile against an installed 2.0.x C library instead of a resident 2.1 (or later) C library, the linking process needs to be adjusted. Section 6 (Compiling with the non-primary libc) of the glibc2 How-To is most helpful here. Here is the process as described (abbreviated):
Set new options for gcc:
-nostdinc no standard includes -I/usr/i486-linuxglibc2/include glibc include files -I/usr/lib/gcc-lib/i486-linuxglibc2/2.7.2.2 gcc include files
Set new options for the linker:
-b i486-linuxglibc2 gcc include files
The appropriate gcc invocation would be:
gcc -b i486-linuxglibc2 \ -nostdinc -I/usr/i486-linuxglibc2/include \ -I/usr/lib/gcc-lib/i486-linuxglibc2/2.7.2.2/include "$@"
This assumes that the appropriate directories are used in place of
i486-linuxglibc2.
In compiling the kernel, the C libraries (and thus the gcc used) do not matter. Any version of Linux, with any version of the gcc C compiler should be able to compile the kernel.
The recent release of Red Hat Linux 7.0 is not recommended for compiling the Linux kernel; the main developer of Linux, Linus Torvalds, is on record as saying that Red Hat Linux 7.0 is unfit for development. This is because the released gcc is a development version — the gcc people do not recommend its use for production work. It has also been suggested that Red 7.0 came with a development version of the C libraries.
Red Hat Linux 6 offers the ability to compile against the 2.0 libraries used for Red Hat 5.2. The full details are at http://www.redhat.com/support/wpapers/glibccompat/ and are quite complete. To begin, several RPMs are needed:
| compat-binutils-5.2 |
| compat-glibc-5.2 |
| compat-libs-5.2 |
| compat-egcs-5.2 |
| compat-egcs-c++-5.2[6] |
Then compiling the desired code will be a matter of using the correct gcc to compile. By default, gcc will be the system's original gcc; if you want to compile with the compatability libraries, you'll need to use the i386-glibc20-linux-gcc compiler located in /usr/bin. Be careful that you do not find yourself using the standard gcc (also called i386-redhat-linux-gcc). By defining a link to /usr/bin/i386-glibc20-linux-gcc in the directory /usr/i386-glibc20-linux/bin directory (using the name gcc) and defining the path appropriately, this problem can be avoided, since gcc will be searched in /usr/i386-glibc20-linux/bin first and found there instead of finding /usr/bin/gcc.
The full details on using Red Hat 6 to compile C programs and C++ programs for glibc 2.0 under Red Hat Linux 6.0 are included on the web site.
In short, compiling a C program entails two things: making sure that the special gcc binary is used, and also that the specially made utilities are used (by setting the PATH). For many projects, the following program will be enough (if used to run other programs like make or configure):
#!/bin/ksh env PATH=/usr/i386-glibc20-linux/bin:$PATH \ CC=i386-glibc20-linux-gcc "$@"
When using make with the compatabile libraries, it is a good idea to use the -e option: this forces the CC environment variable to override any CC variable contained in the Makefile. In this situation, this is a Good Thing.
However, there are many problems that will appear while using a system such as this. It is still easier and less prone to errors if a complete environment such as Debian 2.1 is used.
Compiling a C++ program requires more steps than detailed here; see Red Hat's web site for that information.
Compiling with an arbitrary distribution can actually be done, if there is enough disk space [7]. The steps are these:
Install Debian 2.1 Slink onto a different partition (or even another system entirely) — including all C compilers, libraries, headers, kernel source, et al as needed for development. Slink could probably be installed into about 400M or 500M of disk space — in today's environment of 10G disks and bigger, this is probably not a big problem.
Reboot into the prefered distribution (Red Hat, Mandrake, Slackware, etc.).
Mount the Debian partition under a selected mount point (perhaps /debian). Note that this does not require the disk to be on the same physical system; it could be mounted via NFS.
Perform a chroot to do the compiling — this makes the Debian environment the operating environment for that shell. It may be necessary to also mount the /proc filesystem to /debian/proc — /however, it is not necessary to operate in the Debian environment, only to compile in it. Note too that /proc can be mounted multiple times — so this is not a problem.
Some interesting things to note in particular:
A standalone running Debian system could be mounted via NFS onto another system.
Remember to mount /proc if all operations will be done in the Debian enviroment.
Scripts could be written to force utilities such as make, gcc, or configure to run in a chroot-ed environment.
The Debian gcc compiler that is provided with the Debian Slink system reportedly may have bugs which result in binaries that produce Segmentation Violations.
These are actual examples of source code being compiled and generating problems. This will provide some ideas in how to solve problems with sources that won't compile. Many of the problems are unique to using the older headers instead of the newer ones.
In this case, the problem turned out to be something more than just header files. In compiling ess (echo Security Scanner), several problems were seen. They will be taken one at a time.
tcp_gen.c: In function `tcp_gen': tcp_gen.c:47: structure has no member named `th_sport' tcp_gen.c:48: structure has no member named `th_dport' tcp_gen.c:50: structure has no member named `th_seq' tcp_gen.c:51: structure has no member named `th_ack' tcp_gen.c:56: structure has no member named `th_off' tcp_gen.c:57: structure has no member named `th_x2' tcp_gen.c:60: structure has no member named `th_win' tcp_gen.c:62: structure has no member named `th_flags' tcp_gen.c:62: `TH_SYN' undeclared (first use this function) tcp_gen.c:62: (Each undeclared identifier is reported only once tcp_gen.c:62: for each function it appears in.) tcp.c: In function `syn2port': tcp.c:95: structure has no member named `th_sum'
This is apparently because there is no TH_SYN defined in the header files. So the first step is to determine where TH_SYN is defined in the Linux header files. Looking for the definition turns this up:
# cd /usr/i386-glibc20-linux/include # grep -l "TH_SYN" * */* 2>/dev/null netinet/tcp.h
Notice the directory used for the include files; if
/usr/include is used instead, problems
will result if the installed version of glibc isn't 2.0. In most
modern Red Hat distributions, glibc 2.0 is not used.
To fix these errors, it is necessary to find out why the desired files are not being included. This may be because the proper include directory is not being included, or because the proper subdirectory is not being included. The appropriate source files must be examined to see. Checking tcp_gen.c turns up this portion of code:
#if defined(LINUX) #include <linux/tcp.h> #else #include <netinet/tcp.h> #endif
Presumably (since this is a Linux system) LINUX is defined, and thus the header file is being looked for in linux/tcp.h and not netinet/tcp.h. Changing the source code to use netinet/tcp.h doesn't help.
Next, check netinet/tcp.h and see what the actual definition or usage of TH_SYN was. It appears within a #ifdef ... #else ... #endif block — so it would appear it depends on __FAVOR_BSD being set (which is the variable which is tested). So edit the Makefile of the source to reflect this — since this program uses GNU autoconfigure, the proper file to edit is actually Makefile.in (or changes will be lost after a configure is run!). Thus, a line is changed to define __FAVOR_BSD:
CC = @CC@ @DEFS@ -D_REENTRANT -DLINUX -D__FAVOR_BSD
Now, rerun configure and check the results. In this case, nothing's changed. In the file netinet/tcp.h, there is another #ifdef ... #endif block which prevents defining things more than once. Perhaps this is the problem. To test for it, change the source code so netinet/tcp.h is used early on.
Making this change does not fix it. However, checking the Makefile (again) makes one realize that in this source tree, the variable CC actually holds some of the options to gcc (!), and CC is being replaced by only the alternate C compiler without options. Recompiling without the -e option to make (which causes the CC variable to be overridden) — and this time it seems to work. It also solves several other problems at the same time, and now results in a working binary.
Compiling axfer generates this error:
i386-glibc20-linux-gcc -g -O2 -o axfr axfr.o error.o getopt.o getopt1.o getaxfr.o /usr/lib/libresolv.a zlib-1.1.2/libz.a /usr/lib/libresolv.a(res_send.o): In function `__res_send': /usr/src/bs/BUILD/glibc-2.1.3/resolv/res_send.c:642: undefined reference to `__poll' collect2: ld returned 1 exit status
A important piece of information is here: the reference to /usr/src/bs/BUILD/glibc-2.1.3. It seems likely that /usr/lib/libresolv.a (the resolver library — static version) should not be used, but rather one in /usr/i386-glibc20-linux/lib. Checking there:
# ls -1 /usr/i386-glibc20-linux/lib/*resolv* /usr/i386-glibc20-linux/lib/libresolv-2.0.7.so /usr/i386-glibc20-linux/lib/libresolv.so /usr/i386-glibc20-linux/lib/libresolv.so.2
Note that these are dynamic libraries, not static. Since these do exist here, it is necessary to check whether the /usr/i386-glibc20-linux/lib libraries are being checked for first, before others. Since this source tree uses autoconfig, the file Makefile.in needs to be checked instead of Makefile. Checking the file, a hard-coded reference to /usr/lib/libresolv.a is found in two lines:
LDADD = /usr/lib/libresolv.a zlib-1.1.2/libz.a axfr_DEPENDENCIES = /usr/lib/libresolv.a zlib-1.1.2/libz.a
It is also worth noting that the version of zlib is also hard-coded here. This is almost okay, since it is included in the distribution; however, it means that a more recent version of the library will not be used.
The references to /usr/lib/libresolv.a must be removed. Since these variables appear to be for use with the linker and only specify static libraries, and the dynamic libresolv library is included on most systems, including LRP systems, the library reference is simply removed from the line.
Now there are errors like (abbreviated):
i386-glibc20-linux-gcc -g -O2 -o axfr axfr.o error.o getopt.o getopt1.o getaxfr.o zlib-1.1.2/libz.a getaxfr.o: In function `gzp_fqname': /pub/oxygen/src/net/diags/axfer/axfr-0.5.2/getaxfr.c:86: undefined reference to `__p_fqnname' getaxfr.o: In function `short_prr': /pub/oxygen/src/net/diags/axfer/axfr-0.5.2/getaxfr.c:231: undefined reference to `__p_fqnname' getaxfr.o: In function `do_rrset': /pub/oxygen/src/net/diags/axfer/axfr-0.5.2/getaxfr.c:395: undefined reference to `__dn_skipname' /pub/oxygen/src/net/diags/axfer/axfr-0.5.2/getaxfr.c:399: undefined reference to `_getshort' getaxfr.o: In function `setnslistfordomain': /pub/oxygen/src/net/diags/axfer/axfr-0.5.2/getaxfr.c:432: undefined reference to `res_mkquery' /pub/oxygen/src/net/diags/axfer/axfr-0.5.2/getaxfr.c:438: getaxfr.o: In function `sp_fqnname': /pub/oxygen/src/net/diags/axfer/axfr-0.5.2/getaxfr.c:517: undefined reference to `dn_expand' getaxfr.o: In function `printZone': /pub/oxygen/src/net/diags/axfer/axfr-0.5.2/getaxfr.c:632: undefined reference to `res_mkquery' getaxfr.o: In function `printRR': /pub/oxygen/src/net/diags/axfer/axfr-0.5.2/getaxfr.c:831: undefined reference to `__dn_skipname'
First thing is to find out where these functions are defined. The library libresolv is immediately suspect, since we were working with it — so start there:
# cd /usr/i386-glibc20-linux/lib # nm libresolv.so | grep getshort 000036e0 T _getshort
Doesn't look like libresolv is being referenced. Check Makefile.in and make sure there is a -lresolv option to the C compiler during its link phase. According to make's output there isn't; so add it (to LDFLAGS in this case). It also turns out that axfr is built using axfr_LDFLAGS which does not contain LDFLAGS, so adjust that.
This time it compiles just fine. The libresolv.so dynamic library is referenced, and all symbols are resolved.
[1] This is changing. The latest Oxygen version uses glibc 2.1.3.
[2] Bill
Suethoz <[email protected]> (posting to LRP-dev, 23
October 2000)
[3] Stock LRP includes a stripped-down ncurses v3; Oxygen doesn't use ncurses by default. However, the standard ncurses used for Oxygen is currently ncurses 5.2.
[4] Part of e2fsprogs.
[5] Part of e2fsprogs.
[6] Only needed for compiling C++ programs.
[7] From a discussion on the LRP mailing list.