@node Invoking gnulib-tool
@chapter Invoking gnulib-tool
+@c Copyright (C) 2005-2010 Free Software Foundation, Inc.
+
+@c Permission is granted to copy, distribute and/or modify this document
+@c under the terms of the GNU Free Documentation License, Version 1.3 or
+@c any later version published by the Free Software Foundation; with no
+@c Invariant Sections, with no Front-Cover Texts, and with no Back-Cover
+@c Texts. A copy of the license is included in the ``GNU Free
+@c Documentation License'' file as part of this distribution.
+
@pindex gnulib-tool
@cindex invoking @command{gnulib-tool}
-@command{gnulib-tool} is the way to import Gnulib modules. It is also
-possible to borrow Gnulib modules in a package without using
-@command{gnulib-tool}, relying only on the metainformation stored in
-the @file{modules/*} files, but with a growing number of modules this
-becomes tedious. @command{gnulib-tool} simplifies the management of
-source files, @file{Makefile.am}s and @file{configure.ac} in packages
-incorporating Gnulib modules.
+The @command{gnulib-tool} command is the recommended way to import
+Gnulib modules. It is possible to borrow Gnulib modules in a package
+without using @command{gnulib-tool}, relying only on the
+meta-information stored in the @file{modules/*} files, but with a
+growing number of modules this becomes tedious. @command{gnulib-tool}
+simplifies the management of source files, @file{Makefile.am}s and
+@file{configure.ac} in packages incorporating Gnulib modules.
+
+@file{gnulib-tool} is not installed in a standard directory that is
+contained in the @code{PATH} variable. It needs to be run directly in
+the directory that contains the Gnulib source code. You can do this
+either by specifying the absolute filename of @file{gnulib-tool}, or
+you can also use a symbolic link from a place inside your @code{PATH}
+to the @file{gnulib-tool} file of your preferred and most up-to-date
+Gnulib checkout, like this:
+@smallexample
+$ ln -s $HOME/gnu/src/gnulib.git/gnulib-tool $HOME/bin/gnulib-tool
+@end smallexample
-Run @samp{gnulib-tool --help}. To get familiar with @command{gnulib-tool},
-you can also try some commands with the option @samp{--dry-run}; then
-@code{gnulib-tool} will only report which actions it would perform in a
-real run.
+Run @samp{gnulib-tool --help} for information. To get familiar with
+@command{gnulib-tool} without affecting your sources, you can also try
+some commands with the option @samp{--dry-run}; then
+@code{gnulib-tool} will only report which actions it would perform in
+a real run without changing anything.
@menu
+* Which modules?:: Determining the needed set of Gnulib modules
* Initial import:: First import of Gnulib modules.
* Modified imports:: Changing the import specification.
* Simple update:: Tracking Gnulib development.
-* CVS Issues:: Integration with CVS.
+* Source changes:: Impact of Gnulib on your source files.
+* gettextize and autopoint:: Caveat: @code{gettextize} and @code{autopoint} users!
+* Localization:: Handling Gnulib's own message translations.
+* VCS Issues:: Integration with Version Control Systems.
+* Unit tests:: Bundling the unit tests of the Gnulib modules.
@end menu
+@node Which modules?
+@section Finding modules
+@cindex Finding modules
+
+There are three ways of finding the names of Gnulib modules that you can use
+in your package:
+
+@itemize
+@item
+You have the complete module list, sorted according to categories, in
+@url{http://www.gnu.org/software/gnulib/MODULES.html}.
+
+@item
+If you are looking for a particular POSIX header or function replacement,
+look in the chapters @ref{Header File Substitutes} and
+@ref{Function Substitutes}. For headers and functions that are provided by
+Glibc but not standardized by POSIX, look in the chapters
+@ref{Glibc Header File Substitutes} and @ref{Glibc Function Substitutes}.
+
+@item
+If you have already found the source file in Gnulib and are looking for the
+module that contains this source file, you can use the command
+@samp{gnulib-tool --find @var{filename}}.
+@end itemize
+
+
@node Initial import
@section Initial import
@cindex initial import
-Gnulib assumes your project uses Autoconf and Automake. Invoking
-@samp{gnulib-tool --import} will copy source files, create a
+Gnulib assumes that your project uses Autoconf. When using Gnulib, you
+will need to have Autoconf and Automake among your build tools. Note that
+while the use of Automake in your project's top level directory is an
+easy way to fulfil the Makefile conventions of the GNU coding standards,
+Gnulib does not require it. But when you use Gnulib, Automake will be
+used at least in a subdirectory of your project.
+
+Invoking @samp{gnulib-tool --import} will copy source files, create a
@file{Makefile.am} to build them, generate a file @file{gnulib-comp.m4} with
Autoconf M4 macro declarations used by @file{configure.ac}, and generate
a file @file{gnulib-cache.m4} containing the cached specification of how
Our example will be a library that uses Autoconf, Automake and
Libtool. It calls @code{strdup}, and you wish to use gnulib to make
-the package portable to C89 (which doesn't have @code{strdup}).
+the package portable to C89 and C99 (which don't have @code{strdup}).
@example
~/src/libfoo$ gnulib-tool --import strdup
Module list with included dependencies:
+ absolute-header
+ extensions
strdup
+ string
File list:
+ lib/dummy.c
lib/strdup.c
- lib/strdup.h
- m4/onceonly_2_57.m4
+ lib/string.in.h
+ m4/absolute-header.m4
+ m4/extensions.m4
+ m4/gnulib-common.m4
m4/strdup.m4
-Copying file m4/gnulib-tool.m4
-Copying file m4/onceonly_2_57.m4
+ m4/string_h.m4
+Creating directory ./lib
+Creating directory ./m4
+Copying file lib/dummy.c
Copying file lib/strdup.c
-Copying file lib/strdup.h
+Copying file lib/string.in.h
+Copying file m4/absolute-header.m4
+Copying file m4/extensions.m4
+Copying file m4/gnulib-common.m4
+Copying file m4/gnulib-tool.m4
Copying file m4/strdup.m4
+Copying file m4/string_h.m4
Creating lib/Makefile.am
Creating m4/gnulib-cache.m4
Creating m4/gnulib-comp.m4
Finished.
You may need to add #include directives for the following .h files.
- #include "strdup.h"
+ #include <string.h>
Don't forget to
- add "lib/Makefile" to AC_CONFIG_FILES in ./configure.ac,
neither is specified, the current directory is assumed.
@code{gnulib-tool} can make symbolic links instead of copying the
-source files. Use the @samp{--symbolic} (or @samp{-s} for short) option
-to do this.
+source files. The option to specify for this is @samp{--symlink}, or
+@samp{-s} for short. This can be useful to save a few kilobytes of disk
+space. But it is likely to introduce bugs when @code{gnulib} is updated;
+it is more reliable to use @samp{gnulib-tool --update} (see below)
+to update to newer versions of @code{gnulib}. Furthermore it requires
+extra effort to create self-contained tarballs, and it may disturb some
+mechanism the maintainer applies to the sources. For these reasons,
+this option is generally discouraged.
@code{gnulib-tool} will overwrite any pre-existing files, in
particular @file{Makefile.am}. Unfortunately, separating the
Some Gnulib modules depend on other Gnulib modules. @code{gnulib-tool}
will automatically add the needed modules as well; you need not list
-them explicitly. @code{gnulib-tool} will also memoize which dependent
+them explicitly. @code{gnulib-tool} will also memorize which dependent
modules it has added, so that when someday a dependency is dropped, the
implicitly added module is dropped as well (unless you have explicitly
requested that module).
-If you want to cut a dependency, i.e. not add a module although one of
+If you want to cut a dependency, i.e., not add a module although one of
your requested modules depends on it, you may use the option
@samp{--avoid=@var{module}} to do so. Multiple uses of this option are
possible. Of course, you will then need to implement the same interface
A few manual steps are required to finish the initial import.
@code{gnulib-tool} printed a summary of these steps.
-First, you need to make sure Autoconf can find the macro definitions
-in @file{gnulib-comp.m4}. Use the @code{ACLOCAL_AMFLAGS} specifier in your
-top-level @file{Makefile.am} file, as in:
+First, you must ensure Autoconf can find the macro definitions in
+@file{gnulib-comp.m4}. Use the @code{ACLOCAL_AMFLAGS} specifier in
+your top-level @file{Makefile.am} file, as in:
@example
ACLOCAL_AMFLAGS = -I m4
The core part of the gnulib checks are done by the macro
@code{gl_INIT}. Place it further down in the file, typically where
-you normally check for header files or functions. For example:
+you normally check for header files or functions. It must come after
+other checks which may affect the compiler invocation, such as
+@code{AC_MINIX}. For example:
@example
...
@example
...
-AM_CPPFLAGS = -I$(top_srcdir)/lib -I$(top_builddir)/lib
+AM_CPPFLAGS = -I$(top_builddir)/lib -I$(top_srcdir)/lib
...
-LIBADD = lib/libgnu.a
+LDADD = lib/libgnu.a
...
@end example
Don't forget to @code{#include} the various header files. In this
-example, you would need to make sure that @samp{#include "strdup.h"}
+example, you would need to make sure that @samp{#include <string.h>}
is evaluated when compiling all source code files, that want to make
use of @code{strdup}.
-When an include file is provided by Gnulib
-you shouldn't try to include the corresponding system header files
-yourself, but let the gnulib header file do it. The ordering
-of the definition for some symbols may be significant; the Gnulib
-header files take care of that.
+In the usual case where Autoconf is creating a @file{config.h} file,
+you should include @file{config.h} first, before any other include
+file. That way, for example, if @file{config.h} defines
+@samp{restrict} to be the empty string on a pre-C99 host, or a macro
+like @samp{_FILE_OFFSET_BITS} that affects the layout of data
+structures, the definition is consistent for all include files.
+Also, on some platforms macros like @samp{_FILE_OFFSET_BITS} and
+@samp{_GNU_SOURCE} may be ineffective, or may have only a limited
+effect, if defined after the first system header file is included.
+
+Finally, note that you can not use @code{AC_LIBOBJ} or
+@code{AC_REPLACE_FUNCS} in your @file{configure.ac} and expect the
+resulting object files to be automatically added to @file{lib/libgnu.a}.
+This is because your @code{AC_LIBOBJ} and @code{AC_REPLACE_FUNCS} invocations
+from @file{configure.ac} augment a variable @code{@@LIBOBJS@@} (and/or
+@code{@@LTLIBOBJS@@} if using Libtool), whereas @file{lib/libgnu.a}
+is built from the contents of a different variable, usually
+@code{@@gl_LIBOBJS@@} (or @code{@@gl_LTLIBOBJS@@} if using Libtool).
-For example, to use the @code{time_r} gnulib module you should
-use include header file provided by the gnulib, and so
-@samp{#include "time_r.h"}, but you shouldn't explicitly
-@samp{#include <time.h>} as it is already done in @file{time_r.h}
-before the redefinition of some symbols.
@node Modified imports
@section Modified imports
command line argument.
@item gl_SOURCE_BASE
-The argument is the relative pathname of the directory containing the gnulib
+The argument is the relative file name of the directory containing the gnulib
source files (mostly *.c and *.h files). Corresponds to the
@samp{--source-base} command line argument.
@item gl_M4_BASE
-The argument is the relative pathname of the directory containing the gnulib
+The argument is the relative file name of the directory containing the gnulib
M4 macros (*.m4 files). Corresponds to the @samp{--m4-base} command line
argument.
@item gl_TESTS_BASE
-The argument is the relative pathname of the directory containing the gnulib
+The argument is the relative file name of the directory containing the gnulib
unit test files. Corresponds to the @samp{--tests-base} command line argument.
@item gl_LIB
@samp{--lib} command line argument.
@item gl_LGPL
-The presence of this macro corresponds to the @samp{--lgpl} command line
-argument. It takes no arguments.
+The presence of this macro without arguments corresponds to the @samp{--lgpl}
+command line argument. The presence of this macro with an argument (whose
+value must be 2 or 3) corresponds to the @samp{--lgpl=@var{arg}} command line
+argument.
@item gl_LIBTOOL
The presence of this macro corresponds to the @samp{--libtool} command line
-argument. It takes no arguments.
+argument and to the absence of the @samp{--no-libtool} command line argument.
+It takes no arguments.
@item gl_MACRO_PREFIX
The argument is the prefix to use for macros in the @file{gnulib-comp.m4}
file. Corresponds to the @samp{--macro-prefix} command line argument.
@end table
+
@node Simple update
@section Simple update
$ gnulib-tool --import
@end smallexample
+@noindent
This will create, update or remove files, as needed.
-@node CVS Issues
-@section CVS Issues
+Note: From time to time, changes are made in Gnulib that are not backward
+compatible. When updating to a more recent Gnulib, you should consult
+Gnulib's @file{NEWS} file to check whether the incompatible changes affect
+your project.
+
+
+@node Source changes
+@section Changing your sources for use with Gnulib
+
+Gnulib contains some header file overrides. This means that when building
+on systems with deficient header files in @file{/usr/include/}, it may create
+files named @file{string.h}, @file{stdlib.h}, @file{stdint.h} or similar in
+the build directory. In the other source directories of your package you
+will usually pass @samp{-I} options to the compiler, so that these Gnulib
+substitutes are visible and take precedence over the files in
+@file{/usr/include/}.
+
+These Gnulib substitute header files rely on @file{<config.h>} being
+already included. Furthermore @file{<config.h>} must be the first include
+in every compilation unit. This means that to @emph{all your source files}
+and likely also to @emph{all your tests source files} you need to add an
+@samp{#include <config.h>} at the top. Which source files are affected?
+Exactly those whose compilation includes a @samp{-I} option that refers to
+the Gnulib library directory.
+
+This is annoying, but inevitable: On many systems, @file{<config.h>} is
+used to set system dependent flags (such as @code{_GNU_SOURCE} on GNU systems),
+and these flags have no effect after any system header file has been included.
+
+
+@node gettextize and autopoint
+@section Caveat: @code{gettextize} and @code{autopoint} users
+
+@cindex gettextize, caveat
+@cindex autopoint, caveat
+The programs @code{gettextize} and @code{autopoint}, part of
+GNU @code{gettext}, import or update the internationalization infrastructure.
+Some of this infrastructure, namely ca.@: 20 autoconf macro files and the
+@file{config.rpath} file, is also contained in Gnulib and may be imported
+by @code{gnulib-tool}. The use of @code{gettextize} or @code{autopoint}
+will therefore overwrite some of the files that @code{gnulib-tool} has
+imported, and vice versa.
+
+Avoiding to use @code{gettextize} (manually, as package maintainer) or
+@code{autopoint} (as part of a script like @code{autoreconf} or
+@code{autogen.sh}) is not the solution: These programs also import the
+infrastructure in the @file{po/} and optionally in the @file{intl/} directory.
+
+The copies of the conflicting files in Gnulib are more up-to-date than
+the copies brought in by @code{gettextize} and @code{autopoint}. When a
+new @code{gettext} release is made, the copies of the files in Gnulib will
+be updated immediately.
+
+The solution is therefore:
+
+@enumerate
+@item
+When you run @code{gettextize}, always use the @code{gettextize} from the
+matching GNU gettext release. For the most recent Gnulib checkout, this is
+the newest release found on @url{http://ftp.gnu.org/gnu/gettext/}. For an
+older Gnulib snapshot, it is the release that was the most recent release
+at the time the Gnulib snapshot was taken. Then, after @code{gettextize},
+invoke @code{gnulib-tool}.
+
+@item
+When a script of yours run @code{autopoint}, invoke @code{gnulib-tool}
+afterwards.
+
+@item
+If you get an error message like
+@code{*** error: gettext infrastructure mismatch:
+using a Makefile.in.in from gettext version ...
+but the autoconf macros are from gettext version ...},
+it means that a new GNU gettext release was made, and its autoconf macros
+were integrated into Gnulib and now mismatch the @file{po/} infrastructure.
+In this case, fetch and install the new GNU gettext release and run
+@code{gettextize} followed by @code{gnulib-tool}.
+@end enumerate
+
+
+@node Localization
+@section Handling Gnulib's own message translations
+
+Gnulib provides some functions that emit translatable messages using GNU
+@code{gettext}. The @samp{gnulib} domain at the
+@url{http://translationproject.org/, Translation Project} collects
+translations of these messages, which you should incorporate into your
+own programs.
+
+There are two basic ways to achieve this. The first, and older, method
+is to list all the source files you use from Gnulib in your own
+@file{po/POTFILES.in} file. This will cause all the relevant
+translatable strings to be included in your POT file. When you send
+this POT file to the Translation Project, translators will normally fill
+in the translations of the Gnulib strings from their ``translation
+memory'', and send you back updated PO files.
+
+However, this process is error-prone: you might forget to list some
+source files, or the translator might not be using a translation memory
+and provide a different translation than another translator, or the
+translation might not be kept in sync between Gnulib and your package.
+It is also slow and causes substantial extra work, because a human
+translator must be in the loop for each language and you will need to
+incorporate their work on request.
+
+For these reasons, a new method was designed and is now recommended. If
+you pass the @code{--po-base=@var{directory}} and @code{--po-domain=@var{domain}}
+options to @code{gnulib-tool}, then @code{gnulib-tool} will create a
+separate directory with its own @file{POTFILES.in}, and fetch current
+translations directly from the Translation Project (using
+@command{rsync} or @command{wget}, whichever is available).
+The POT file in this directory will be called
+@file{@var{domain}-gnulib.pot}, depending on the @var{domain} you gave to the
+@code{--po-domain} option (typically the same as the package name).
+This causes these translations to reside in a separate message domain,
+so that they do not clash either with the translations for the main part
+of your package nor with those of other packages on the system that use
+possibly different versions of Gnulib.
+When you use these options, the functions in Gnulib are built
+in such a way that they will always use this domain regardless of the
+default domain set by @code{textdomain}.
+
+In order to use this method, you must -- in each program that might use
+Gnulib code -- add an extra line to the part of the program that
+initializes locale-dependent behavior. Where you would normally write
+something like:
+
+@example
+@group
+ setlocale (LC_ALL, "");
+ bindtextdomain (PACKAGE, LOCALEDIR);
+ textdomain (PACKAGE);
+@end group
+@end example
+
+@noindent
+you should add an additional @code{bindtextdomain} call to inform
+gettext of where the MO files for the extra message domain may be found:
+
+@example
+@group
+ bindtextdomain (PACKAGE "-gnulib", LOCALEDIR);
+@end group
+@end example
+
+(This example assumes that the @var{domain} that you specified
+to @code{gnulib-tool} is the same as the value of the @code{PACKAGE}
+preprocessor macro.)
+
+Since you do not change the @code{textdomain} call, the default message
+domain for your program remains the same and your own use of @code{gettext}
+functions will not be affected.
+
+
+@node VCS Issues
+@section Issues with Version Control Systems
+
+If a project stores its source files in a version control system (VCS),
+such as CVS, SVN, or Git, one needs to decide which files to commit.
All files created by @code{gnulib-tool}, except @file{gnulib-cache.m4},
should be treated like generated source files, like for example a
@file{parser.c} file is generated from @file{parser.y}.
-In projects which commit all source files, whether generated or not, into
-CVS, the @code{gnulib-tool} generated files should all be committed.
+@itemize
-In projects which customarily omit from the CVS all files that generated
+@item
+In projects which commit all source files, whether generated or not, into
+their VCS, the @code{gnulib-tool} generated files should all be committed.
+In this case, you also pass the option @samp{--no-vc-files} to
+@code{gnulib-tool}.
+
+Gnulib also contains files generated by @command{make} (and removed by
+@code{make clean}), using information determined by @command{configure}.
+They should not be checked into the VCS, but instead added to
+@file{.gitignore} or @file{.cvsignore}.
+When you have a Gnulib source file of the form @file{lib/foo.in.h}, the
+corresponding @file{lib/foo.h} is such a file.
+
+@item
+In projects which customarily omit from their VCS all files that are generated
from other source files, all these files and directories would not be
-added into CVS. The only file that must be added to CVS is
+added into the VCS. The only file that must be added to the VCS is
@file{gnulib-cache.m4} in the M4 macros directory. Also, the script for
-restoring files not in CVS, customarily called @file{autogen.sh} or
+restoring files not in the VCS, customarily called @file{autogen.sh} or
@file{bootstrap.sh}, will typically contain the statement for restoring
the omitted files:
but it does not offer the possibility to change the way Gnulib is used.
Also it does not report in the ChangeLogs the files that it had to add
because they were missing.
+
+@end itemize
+
+
+@node Unit tests
+@section Bundling the unit tests of the Gnulib modules
+
+You can bundle the unit tests of the Gnulib modules together with your
+package, through the @samp{--with-tests} option. Together with
+@samp{--with-tests}, you also specify the directory for these tests
+through the @samp{--tests-base} option. Of course, you need to add this
+directory to the @code{SUBDIRS} variable in the @code{Makefile.am} of
+the parent directory.
+
+The advantage of having the unit tests bundled is that when your program
+has a problem on a particular platform, running the unit tests may help
+determine quickly if the problem is on Gnulib's side or on your package's
+side. Also, it helps verifying Gnulib's portability, of course.
+
+The unit tests will be compiled and run when the user runs @samp{make check}.
+When the user runs only @samp{make}, the unit tests will not be compiled.
+
+In the @code{SUBDIRS} variable, it is useful to put the Gnulib tests directory
+after the directory containing the other tests, not before:
+
+@smallexample
+SUBDIRS = gnulib-lib src man tests gnulib-tests
+@end smallexample
+
+@noindent
+This will ensure that on platforms where there are test failures in either
+directory, users will see and report the failures from the tests of your
+program.
+
+Note: In packages which use more than one invocation of @code{gnulib-tool}
+in the scope of the same @code{configure.ac}, you cannot use
+@samp{--with-tests}. You will have to use a separate @code{configure.ac}
+in this case.