@node Invoking gnulib-tool
@chapter Invoking gnulib-tool
-@c Copyright (C) 2005, 2006, 2007 Free Software Foundation, Inc.
+@c Copyright (C) 2005-2010 Free Software Foundation, Inc.
@c Permission is granted to copy, distribute and/or modify this document
-@c under the terms of the GNU Free Documentation License, Version 1.2 or
+@c under the terms of the GNU Free Documentation License, Version 1.3 or
@c any later version published by the Free Software Foundation; with no
@c Invariant Sections, with no Front-Cover Texts, and with no Back-Cover
@c Texts. A copy of the license is included in the ``GNU Free
simplifies the management of source files, @file{Makefile.am}s and
@file{configure.ac} in packages incorporating Gnulib modules.
+@file{gnulib-tool} is not installed in a standard directory that is
+contained in the @code{PATH} variable. It needs to be run directly in
+the directory that contains the Gnulib source code. You can do this
+either by specifying the absolute filename of @file{gnulib-tool}, or
+you can also use a symbolic link from a place inside your @code{PATH}
+to the @file{gnulib-tool} file of your preferred and most up-to-date
+Gnulib checkout, like this:
+@smallexample
+$ ln -s $HOME/gnu/src/gnulib.git/gnulib-tool $HOME/bin/gnulib-tool
+@end smallexample
+
Run @samp{gnulib-tool --help} for information. To get familiar with
@command{gnulib-tool} without affecting your sources, you can also try
some commands with the option @samp{--dry-run}; then
a real run without changing anything.
@menu
+* Which modules?:: Determining the needed set of Gnulib modules
* Initial import:: First import of Gnulib modules.
* Modified imports:: Changing the import specification.
* Simple update:: Tracking Gnulib development.
-* CVS Issues:: Integration with CVS.
+* Source changes:: Impact of Gnulib on your source files.
+* gettextize and autopoint:: Caveat: @code{gettextize} and @code{autopoint} users!
+* Localization:: Handling Gnulib's own message translations.
+* VCS Issues:: Integration with Version Control Systems.
+* Unit tests:: Bundling the unit tests of the Gnulib modules.
@end menu
+@node Which modules?
+@section Finding modules
+@cindex Finding modules
+
+There are three ways of finding the names of Gnulib modules that you can use
+in your package:
+
+@itemize
+@item
+You have the complete module list, sorted according to categories, in
+@url{http://www.gnu.org/software/gnulib/MODULES.html}.
+
+@item
+If you are looking for a particular POSIX header or function replacement,
+look in the chapters @ref{Header File Substitutes} and
+@ref{Function Substitutes}. For headers and functions that are provided by
+Glibc but not standardized by POSIX, look in the chapters
+@ref{Glibc Header File Substitutes} and @ref{Glibc Function Substitutes}.
+
+@item
+If you have already found the source file in Gnulib and are looking for the
+module that contains this source file, you can use the command
+@samp{gnulib-tool --find @var{filename}}.
+@end itemize
+
+
@node Initial import
@section Initial import
@cindex initial import
-Gnulib assumes your project uses Autoconf and Automake. Invoking
-@samp{gnulib-tool --import} will copy source files, create a
+Gnulib assumes that your project uses Autoconf. When using Gnulib, you
+will need to have Autoconf and Automake among your build tools. Note that
+while the use of Automake in your project's top level directory is an
+easy way to fulfil the Makefile conventions of the GNU coding standards,
+Gnulib does not require it. But when you use Gnulib, Automake will be
+used at least in a subdirectory of your project.
+
+Invoking @samp{gnulib-tool --import} will copy source files, create a
@file{Makefile.am} to build them, generate a file @file{gnulib-comp.m4} with
Autoconf M4 macro declarations used by @file{configure.ac}, and generate
a file @file{gnulib-cache.m4} containing the cached specification of how
File list:
lib/dummy.c
lib/strdup.c
- lib/string_.h
+ lib/string.in.h
m4/absolute-header.m4
m4/extensions.m4
m4/gnulib-common.m4
Creating directory ./m4
Copying file lib/dummy.c
Copying file lib/strdup.c
-Copying file lib/string_.h
+Copying file lib/string.in.h
Copying file m4/absolute-header.m4
Copying file m4/extensions.m4
Copying file m4/gnulib-common.m4
@example
...
-AM_CPPFLAGS = -I$(top_srcdir)/lib -I$(top_builddir)/lib
+AM_CPPFLAGS = -I$(top_builddir)/lib -I$(top_srcdir)/lib
...
LDADD = lib/libgnu.a
...
In the usual case where Autoconf is creating a @file{config.h} file,
you should include @file{config.h} first, before any other include
file. That way, for example, if @file{config.h} defines
-@samp{restrict} to be the empty string on a pre-C99 host, the
-definition is consistent for all include files.
-
-You should include Gnulib-provided headers before system headers,
-so that Gnulib-provided headers can adjust how a system header
-behaves.
-
-A final word of warning: Gnulib currently assumes it will be
-responsible for @emph{all} functions that end up in the Autoconf
-@code{@@LIBOBJS@@} variables (and/or @code{@@LTLIBOBJS@@} if using
-Libtool), e.g., those specified in @code{AC_REPLACE_FUNCS} in your
-@file{configure.ac}. Therefore, if you have any functions which are
-not covered by Gnulib which need that treatment, you have to
-essentially reimplement AC_REPLACE_FUNCS using different names; for an
-example, see the Findutils sources. Perhaps this will be improved in
-the future.
+@samp{restrict} to be the empty string on a pre-C99 host, or a macro
+like @samp{_FILE_OFFSET_BITS} that affects the layout of data
+structures, the definition is consistent for all include files.
+Also, on some platforms macros like @samp{_FILE_OFFSET_BITS} and
+@samp{_GNU_SOURCE} may be ineffective, or may have only a limited
+effect, if defined after the first system header file is included.
+
+Finally, note that you can not use @code{AC_LIBOBJ} or
+@code{AC_REPLACE_FUNCS} in your @file{configure.ac} and expect the
+resulting object files to be automatically added to @file{lib/libgnu.a}.
+This is because your @code{AC_LIBOBJ} and @code{AC_REPLACE_FUNCS} invocations
+from @file{configure.ac} augment a variable @code{@@LIBOBJS@@} (and/or
+@code{@@LTLIBOBJS@@} if using Libtool), whereas @file{lib/libgnu.a}
+is built from the contents of a different variable, usually
+@code{@@gl_LIBOBJS@@} (or @code{@@gl_LTLIBOBJS@@} if using Libtool).
@node Modified imports
@samp{--lib} command line argument.
@item gl_LGPL
-The presence of this macro corresponds to the @samp{--lgpl} command line
-argument. It takes no arguments.
+The presence of this macro without arguments corresponds to the @samp{--lgpl}
+command line argument. The presence of this macro with an argument (whose
+value must be 2 or 3) corresponds to the @samp{--lgpl=@var{arg}} command line
+argument.
@item gl_LIBTOOL
The presence of this macro corresponds to the @samp{--libtool} command line
file. Corresponds to the @samp{--macro-prefix} command line argument.
@end table
+
@node Simple update
@section Simple update
$ gnulib-tool --import
@end smallexample
+@noindent
This will create, update or remove files, as needed.
-@node CVS Issues
-@section CVS Issues
+Note: From time to time, changes are made in Gnulib that are not backward
+compatible. When updating to a more recent Gnulib, you should consult
+Gnulib's @file{NEWS} file to check whether the incompatible changes affect
+your project.
+
+
+@node Source changes
+@section Changing your sources for use with Gnulib
+
+Gnulib contains some header file overrides. This means that when building
+on systems with deficient header files in @file{/usr/include/}, it may create
+files named @file{string.h}, @file{stdlib.h}, @file{stdint.h} or similar in
+the build directory. In the other source directories of your package you
+will usually pass @samp{-I} options to the compiler, so that these Gnulib
+substitutes are visible and take precedence over the files in
+@file{/usr/include/}.
+
+These Gnulib substitute header files rely on @file{<config.h>} being
+already included. Furthermore @file{<config.h>} must be the first include
+in every compilation unit. This means that to @emph{all your source files}
+and likely also to @emph{all your tests source files} you need to add an
+@samp{#include <config.h>} at the top. Which source files are affected?
+Exactly those whose compilation includes a @samp{-I} option that refers to
+the Gnulib library directory.
+
+This is annoying, but inevitable: On many systems, @file{<config.h>} is
+used to set system dependent flags (such as @code{_GNU_SOURCE} on GNU systems),
+and these flags have no effect after any system header file has been included.
+
+
+@node gettextize and autopoint
+@section Caveat: @code{gettextize} and @code{autopoint} users
+
+@cindex gettextize, caveat
+@cindex autopoint, caveat
+The programs @code{gettextize} and @code{autopoint}, part of
+GNU @code{gettext}, import or update the internationalization infrastructure.
+Some of this infrastructure, namely ca.@: 20 autoconf macro files and the
+@file{config.rpath} file, is also contained in Gnulib and may be imported
+by @code{gnulib-tool}. The use of @code{gettextize} or @code{autopoint}
+will therefore overwrite some of the files that @code{gnulib-tool} has
+imported, and vice versa.
+
+Avoiding to use @code{gettextize} (manually, as package maintainer) or
+@code{autopoint} (as part of a script like @code{autoreconf} or
+@code{autogen.sh}) is not the solution: These programs also import the
+infrastructure in the @file{po/} and optionally in the @file{intl/} directory.
+
+The copies of the conflicting files in Gnulib are more up-to-date than
+the copies brought in by @code{gettextize} and @code{autopoint}. When a
+new @code{gettext} release is made, the copies of the files in Gnulib will
+be updated immediately.
+
+The solution is therefore:
+
+@enumerate
+@item
+When you run @code{gettextize}, always use the @code{gettextize} from the
+matching GNU gettext release. For the most recent Gnulib checkout, this is
+the newest release found on @url{http://ftp.gnu.org/gnu/gettext/}. For an
+older Gnulib snapshot, it is the release that was the most recent release
+at the time the Gnulib snapshot was taken. Then, after @code{gettextize},
+invoke @code{gnulib-tool}.
+
+@item
+When a script of yours run @code{autopoint}, invoke @code{gnulib-tool}
+afterwards.
+
+@item
+If you get an error message like
+@code{*** error: gettext infrastructure mismatch:
+using a Makefile.in.in from gettext version ...
+but the autoconf macros are from gettext version ...},
+it means that a new GNU gettext release was made, and its autoconf macros
+were integrated into Gnulib and now mismatch the @file{po/} infrastructure.
+In this case, fetch and install the new GNU gettext release and run
+@code{gettextize} followed by @code{gnulib-tool}.
+@end enumerate
+
+
+@node Localization
+@section Handling Gnulib's own message translations
+
+Gnulib provides some functions that emit translatable messages using GNU
+@code{gettext}. The @samp{gnulib} domain at the
+@url{http://translationproject.org/, Translation Project} collects
+translations of these messages, which you should incorporate into your
+own programs.
+
+There are two basic ways to achieve this. The first, and older, method
+is to list all the source files you use from Gnulib in your own
+@file{po/POTFILES.in} file. This will cause all the relevant
+translatable strings to be included in your POT file. When you send
+this POT file to the Translation Project, translators will normally fill
+in the translations of the Gnulib strings from their ``translation
+memory'', and send you back updated PO files.
+
+However, this process is error-prone: you might forget to list some
+source files, or the translator might not be using a translation memory
+and provide a different translation than another translator, or the
+translation might not be kept in sync between Gnulib and your package.
+It is also slow and causes substantial extra work, because a human
+translator must be in the loop for each language and you will need to
+incorporate their work on request.
+
+For these reasons, a new method was designed and is now recommended. If
+you pass the @code{--po-base=@var{directory}} and @code{--po-domain=@var{domain}}
+options to @code{gnulib-tool}, then @code{gnulib-tool} will create a
+separate directory with its own @file{POTFILES.in}, and fetch current
+translations directly from the Translation Project (using
+@command{rsync} or @command{wget}, whichever is available).
+The POT file in this directory will be called
+@file{@var{domain}-gnulib.pot}, depending on the @var{domain} you gave to the
+@code{--po-domain} option (typically the same as the package name).
+This causes these translations to reside in a separate message domain,
+so that they do not clash either with the translations for the main part
+of your package nor with those of other packages on the system that use
+possibly different versions of Gnulib.
+When you use these options, the functions in Gnulib are built
+in such a way that they will always use this domain regardless of the
+default domain set by @code{textdomain}.
+
+In order to use this method, you must -- in each program that might use
+Gnulib code -- add an extra line to the part of the program that
+initializes locale-dependent behavior. Where you would normally write
+something like:
+
+@example
+@group
+ setlocale (LC_ALL, "");
+ bindtextdomain (PACKAGE, LOCALEDIR);
+ textdomain (PACKAGE);
+@end group
+@end example
+
+@noindent
+you should add an additional @code{bindtextdomain} call to inform
+gettext of where the MO files for the extra message domain may be found:
+
+@example
+@group
+ bindtextdomain (PACKAGE "-gnulib", LOCALEDIR);
+@end group
+@end example
+
+(This example assumes that the @var{domain} that you specified
+to @code{gnulib-tool} is the same as the value of the @code{PACKAGE}
+preprocessor macro.)
+
+Since you do not change the @code{textdomain} call, the default message
+domain for your program remains the same and your own use of @code{gettext}
+functions will not be affected.
+
+
+@node VCS Issues
+@section Issues with Version Control Systems
+
+If a project stores its source files in a version control system (VCS),
+such as CVS, SVN, or Git, one needs to decide which files to commit.
All files created by @code{gnulib-tool}, except @file{gnulib-cache.m4},
should be treated like generated source files, like for example a
@item
In projects which commit all source files, whether generated or not, into
-CVS, the @code{gnulib-tool} generated files should all be committed.
+their VCS, the @code{gnulib-tool} generated files should all be committed.
+In this case, you also pass the option @samp{--no-vc-files} to
+@code{gnulib-tool}.
Gnulib also contains files generated by @command{make} (and removed by
-@code{make clean}, using information determined by @command{configure}
-They should not be checked into CVS, but instead added to
-@file{.cvsignore}. When you have a Gnulib source file of the form
-@file{lib/foo_.h}, the corresponding @file{lib/foo.h} is such a file.
-
+@code{make clean}), using information determined by @command{configure}.
+They should not be checked into the VCS, but instead added to
+@file{.gitignore} or @file{.cvsignore}.
+When you have a Gnulib source file of the form @file{lib/foo.in.h}, the
+corresponding @file{lib/foo.h} is such a file.
@item
-In projects which customarily omit from the CVS all files that generated
+In projects which customarily omit from their VCS all files that are generated
from other source files, all these files and directories would not be
-added into CVS. The only file that must be added to CVS is
+added into the VCS. The only file that must be added to the VCS is
@file{gnulib-cache.m4} in the M4 macros directory. Also, the script for
-restoring files not in CVS, customarily called @file{autogen.sh} or
+restoring files not in the VCS, customarily called @file{autogen.sh} or
@file{bootstrap.sh}, will typically contain the statement for restoring
the omitted files:
because they were missing.
@end itemize
+
+
+@node Unit tests
+@section Bundling the unit tests of the Gnulib modules
+
+You can bundle the unit tests of the Gnulib modules together with your
+package, through the @samp{--with-tests} option. Together with
+@samp{--with-tests}, you also specify the directory for these tests
+through the @samp{--tests-base} option. Of course, you need to add this
+directory to the @code{SUBDIRS} variable in the @code{Makefile.am} of
+the parent directory.
+
+The advantage of having the unit tests bundled is that when your program
+has a problem on a particular platform, running the unit tests may help
+determine quickly if the problem is on Gnulib's side or on your package's
+side. Also, it helps verifying Gnulib's portability, of course.
+
+The unit tests will be compiled and run when the user runs @samp{make check}.
+When the user runs only @samp{make}, the unit tests will not be compiled.
+
+In the @code{SUBDIRS} variable, it is useful to put the Gnulib tests directory
+after the directory containing the other tests, not before:
+
+@smallexample
+SUBDIRS = gnulib-lib src man tests gnulib-tests
+@end smallexample
+
+@noindent
+This will ensure that on platforms where there are test failures in either
+directory, users will see and report the failures from the tests of your
+program.
+
+Note: In packages which use more than one invocation of @code{gnulib-tool}
+in the scope of the same @code{configure.ac}, you cannot use
+@samp{--with-tests}. You will have to use a separate @code{configure.ac}
+in this case.