Major update of the "Invoking gnulib-tool" chapter.

[gnulib.git] / doc / gnulib.texi

\input texinfo   @c -*-texinfo-*-
@comment $Id: gnulib.texi,v 1.18 2005-09-19 15:38:33 haible Exp $
@comment %**start of header
@setfilename gnulib.info
@settitle GNU Gnulib
@syncodeindex fn cp
@syncodeindex pg cp
@comment %**end of header

@set UPDATED $Date: 2005-09-19 15:38:33 $

@copying
This manual is for GNU Gnulib (updated @value{UPDATED}),
which is a library of common routines intended to be shared at the
source level.

Copyright @copyright{} 2004, 2005 Free Software Foundation, Inc.

@quotation
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.1 or
any later version published by the Free Software Foundation; with no
Invariant Sections, with the Front-Cover Texts being ``A GNU Manual,''
and with the Back-Cover Texts as in (a) below.  A copy of the
license is included in the section entitled ``GNU Free Documentation
License.''

(a) The FSF's Back-Cover Text is: ``You have freedom to copy and modify
this GNU Manual, like GNU software.  Copies published by the Free
Software Foundation raise funds for GNU development.''
@end quotation
@end copying

@dircategory Software development
@direntry
* Gnulib: (gnulib).             Source files to share among distributions.
@end direntry

@titlepage
@title GNU Gnulib
@subtitle updated @value{UPDATED}
@author @email{bug-gnulib@@gnu.org}
@page
@vskip 0pt plus 1filll
@insertcopying
@end titlepage

@contents

@ifnottex
@node Top
@top GNU Gnulib

@insertcopying
@end ifnottex

@menu
* Gnulib::
* Invoking gnulib-tool::
* Copying This Manual::
* Index::
@end menu


@node Gnulib
@chapter Gnulib

This manual contains some bare-bones documentation, but not much more.
It's mostly been a place to store notes until someone (you?)@ gets
around to writing a coherent manual.

Getting started:

@itemize
@item Gnulib is hosted at Savannah:
      @url{http://savannah.gnu.org/projects/gnulib}.  Get the sources
      through CVS from there.
@item The Gnulib home page:
      @url{http://www.gnu.org/software/gnulib/}.
@end itemize

@menu
* Comments::
* Header files::
* Quoting::
* ctime::
* inet_ntoa::
* Out of memory handling::
* Library version handling::
* Regular expressions::
@end menu


@node Comments
@section Comments

@cindex comments describing functions
@cindex describing functions, locating
Where to put comments describing functions: Because of risk of
divergence, we prefer to keep most function describing comments in
only one place: just above the actual function definition.  Some
people prefer to put that documentation in the .h file.  In any case,
it should appear in just one place unless you can ensure that the
multiple copies will always remain identical.


@node Header files
@section Header files

@cindex double inclusion of header files
@cindex header file include protection
It is a tradition to use CPP tricks to avoid parsing the same header
file more than once, which might cause warnings.  The trick is to wrap
the content of the header file (say, @file{foo.h}) in a block, as in:

@example
#ifndef FOO_H
# define FOO_H
...
body of header file goes here
...
#endif /* FOO_H */
@end example

Whether to use @code{FOO_H} or @code{_FOO_H} is a matter of taste and
style.  The C89 and C99 standards reserve all identifiers that begin with an
underscore and either an uppercase letter or another underscore, for
any use.  Thus, in theory, an application might not safely assume that
@code{_FOO_H} has not already been defined by a library.  On the other
hand, using @code{FOO_H} will likely lead the higher risk of
collisions with other symbols (e.g., @code{KEY_H}, @code{XK_H}, @code{BPF_H},
which are CPP macro constants, or @code{COFF_LONG_H}, which is a CPP
macro function).  Your preference may depend on whether you consider
the header file under discussion as part of the application (which has
its own namespace for CPP symbols) or a supporting library (that
shouldn't interfere with the application's CPP symbol namespace).

@cindex C++ header files
@cindex Header files and C++
Adapting C header files for use in C++ applications can use another
CPP trick, as in:

@example
# ifdef __cplusplus
extern "C"
@{
# endif
...
body of header file goes here
...
# ifdef __cplusplus
@}
# endif
@end example

The idea here is that @code{__cplusplus} is defined only by C++
implementations, which will wrap the header file in an @samp{extern "C"}
block.  Again, whether to use this trick is a matter of taste and
style.  While the above can be seen as harmless, it could be argued
that the header file is written in C, and any C++ application using it
should explicitly use the @samp{extern "C"} block itself.  Your
preference might depend on whether you consider the API exported by
your header file as something available for C programs only, or for C
and C++ programs alike.

@subheading Include ordering

When writing a gnulib module, or even in general, a good way to order
the @samp{#include} directives is the following.

@itemize
@item First comes the #include "..." specifying the module being implemented.
@item Then come all the #include <...> of system or system-replacement headers,
in arbitrary order.
@item Then come all the #include "..." of gnulib and private headers, in
arbitrary order.
@end itemize


@node Quoting
@section Quoting

@cindex Quoting
@findex quote
@findex quotearg

Gnulib provides @samp{quote} and @samp{quotearg} modules to help with
quoting text, such as file names, in messages to the user.  Here's an
example of using @samp{quote}:

@example
#include <quote.h>
 ...
  error (0, errno, _("cannot change owner of %s"), quote (fname));
@end example

This differs from

@example
  error (0, errno, _("cannot change owner of `%s'"), fname);
@end example

@noindent in that @code{quote} escapes unusual characters in
@code{fname}, e.g., @samp{'} and control characters like @samp{\n}.

@findex quote_n
However, a caveat: @code{quote} reuses the storage that it returns.
Hence if you need more than one thing quoted at the same time, you
need to use @code{quote_n}.

@findex quotearg_alloc
Also, the quote module is not suited for multithreaded applications.
In that case, you have to use @code{quotearg_alloc}, defined in the
@samp{quotearg} module, which is decidedly less convenient.


@node ctime
@section ctime
@findex ctime

The @code{ctime} function need not be reentrant, and consequently is
not required to be thread safe.  Implementations of @code{ctime}
typically write the time stamp into static buffer.  If two threads
call @code{ctime} at roughly the same time, you might end up with the
wrong date in one of the threads, or some undefined string.  There is
a re-entrant interface @code{ctime_r}, that take a pre-allocated
buffer and length of the buffer, and return @code{NULL} on errors.
The input buffer should be at least 26 bytes in size.  The output
string is locale-independent.  However, years can have more than 4
digits if @code{time_t} is sufficiently wide, so the length of the
required output buffer is not easy to determine.  Increasing the
buffer size when @code{ctime_r} return @code{NULL} is not necessarily
sufficient. The @code{NULL} return value could mean some other error
condition, which will not go away by increasing the buffer size.

A more flexible function is @code{strftime}.  However, note that it is
locale dependent.


@node inet_ntoa
@section inet_ntoa
@findex inet_ntoa

The @code{inet_ntoa} function need not be reentrant, and consequently
is not required to be thread safe.  Implementations of
@code{inet_ntoa} typically write the time stamp into static buffer.
If two threads call @code{inet_ntoa} at roughly the same time, you
might end up with the wrong date in one of the threads, or some
undefined string.  Further, @code{inet_ntoa} is specific for
@acronym{IPv4} addresses.

A protocol independent function is @code{inet_ntop}.


@node Out of memory handling
@section Out of memory handling

@cindex Out of Memory handling
@cindex Memory allocation failure
The GSS API does not have a standard error code for the out of memory
error condition.  Instead of adding a non-standard error code, this
library has chosen to adopt a different strategy.  Out of memory
handling happens in rare situations, but performing the out of memory
error handling after almost all API function invocations pollute your
source code and might make it harder to spot more serious problems.
The strategy chosen improve code readability and robustness.

@cindex Aborting execution
For most applications, aborting the application with an error message
when the out of memory situation occur is the best that can be wished
for.  This is how the library behaves by default.

@vindex xalloc_fail_func
However, we realize that some applications may not want to have the
GSS library abort execution in any situation.  The GSS library support
a hook to let the application regain control and perform its own
cleanups when an out of memory situation has occured.  The application
can define a function (having a @code{void} prototype, i.e., no return
value and no parameters) and set the library variable
@code{xalloc_fail_func} to that function.  The variable should be
declared as follows.

@example
extern void (*xalloc_fail_func) (void);
@end example

The GSS library will invoke this function if an out of memory error
occurs.  Note that after this the GSS library is in an undefined
state, so you must unload or restart the application to continue call
GSS library functions.  The hook is only intended to allow the
application to log the situation in a special way.  Of course, care
must be taken to not allocate more memory, as that will likely also
fail.


@node Library version handling
@section Library version handling

The module @samp{check-version} can be useful when your gnulib
application is a system library.  You will typically wrap the call to
the @code{check_version} function through a library API, your library
header file may contain:

@example
#define STRINGPREP_VERSION "0.5.18"
...
  extern const char *stringprep_check_version (const char *req_version);
@end example

To avoid ELF symbol collisions with other libraries that use the
@samp{check-version} module, add to @file{config.h} through a
AC_DEFINE something like:

@example
AC_DEFINE(check_version, stringprep_check_version, [Rename check_version.])
@end example

The @code{stringprep_check_version} function will thus be implemented
by the @code{check_version} module.

There are two uses of the interface.  The first is a way to provide
for applications to find out the version number of the library it
uses.  The application may contain diagnostic code such as:

@example
  printf ("Stringprep version: header %s library %s",
          STRINGPREP_VERSION,
          stringprep_check_version (NULL));
@end example

Separating the library and header file version can be useful when
searching for version mismatch related problems.

The second uses is as a rudimentary test of proper library version, by
making sure the application get a library version that is the same, or
newer, than the header file used when building the application.  This
doesn't catch all problems, libraries may change backwards incompatibly
in later versions, but enable applications to require a certain
minimum version before it may proceed.

Typical uses look like:

@example
       /* Check version of libgcrypt. */
       if (!gcry_check_version (GCRYPT_VERSION))
         die ("version mismatch\n");
@end example


@node Regular expressions
@section Regular expressions

Gnulib supports many different types of regular expressions; although
the underlying features are the same or identical, the syntax used
varies.  The descriptions given here for the different types are
generated automatically.

@include regexprops-generic.texi


@node Invoking gnulib-tool
@chapter Invoking gnulib-tool

@pindex gnulib-tool
@cindex invoking @command{gnulib-tool}

@command{gnulib-tool} is the way to import Gnulib modules.  It is also
possible to borrow Gnulib modules in a package without using
@command{gnulib-tool}, relying only on the metainformation stored in
the @file{modules/*} files, but with a growing number of modules this
becomes tedious.  @command{gnulib-tool} simplifies the management of
source files, @file{Makefile.am}s and @file{configure.ac} in packages
incorporating Gnulib modules.

Run @samp{gnulib-tool --help}.  To get familiar with @command{gnulib-tool},
you can also try some commands with the option @samp{--dry-run}; then
@code{gnulib-tool} will only report which actions it would perform in a
real run.

@menu
* Initial import::              First import of Gnulib modules.
* Modified imports::            Changing the import specification.
* Simple update::               Tracking Gnulib development.
* CVS Issues::                  Integration with CVS.
@end menu


@node Initial import
@section Initial import
@cindex initial import

Gnulib assumes your project uses Autoconf and Automake.  Invoking
@samp{gnulib-tool --import} will copy source files, create a
@file{Makefile.am} to build them, generate a file @file{gnulib-comp.m4} with
Autoconf M4 macro declarations used by @file{configure.ac}, and generate
a file @file{gnulib-cache.m4} containing the cached specification of how
Gnulib is used.

Our example will be a library that uses Autoconf, Automake and
Libtool.  It calls @code{strdup}, and you wish to use gnulib to make
the package portable to C89 (which doesn't have @code{strdup}).

@example
~/src/libfoo$ gnulib-tool --import strdup
Module list with included dependencies:
  strdup
File list:
  lib/strdup.c
  lib/strdup.h
  m4/onceonly_2_57.m4
  m4/strdup.m4
Copying file m4/gnulib-tool.m4
Copying file m4/onceonly_2_57.m4
Copying file lib/strdup.c
Copying file lib/strdup.h
Copying file m4/strdup.m4
Creating lib/Makefile.am
Creating m4/gnulib-cache.m4
Creating m4/gnulib-comp.m4
Finished.

You may need to add #include directives for the following .h files.
  #include "strdup.h"

Don't forget to
  - add "lib/Makefile" to AC_CONFIG_FILES in ./configure.ac,
  - mention "lib" in SUBDIRS in Makefile.am,
  - mention "-I m4" in ACLOCAL_AMFLAGS in Makefile.am,
  - invoke gl_EARLY in ./configure.ac, right after AC_PROG_CC,
  - invoke gl_INIT in ./configure.ac.
~/src/libfoo$
@end example

By default, the source code is copied into @file{lib/} and the M4
macros in @file{m4/}.  You can override these paths by using
@code{--source-base=DIRECTORY} and @code{--m4-base=DIRECTORY}.  Some
modules also provide other files necessary for building. These files
are copied into the directory specified by @samp{AC_CONFIG_AUX_DIR} in
@file{configure.ac} or by the @code{--aux-dir=DIRECTORY} option.  If
neither is specified, the current directory is assumed.

@code{gnulib-tool} can make symbolic links instead of copying the
source files.  Use the @samp{--symbolic} (or @samp{-s} for short) option
to do this.

@code{gnulib-tool} will overwrite any pre-existing files, in
particular @file{Makefile.am}.  Unfortunately, separating the
generated @file{Makefile.am} content (for building the gnulib library)
into a separate file, say @file{gnulib.mk}, that could be included
by your handwritten @file{Makefile.am} is not possible, due to how
variable assignments are handled by Automake.

Consequently, it is a good idea to choose directories that are not
already used by your projects, to separate gnulib imported files from
your own files.  This approach is also useful if you want to avoid
conflicts between other tools (e.g., @code{gettextize} that also copy
M4 files into your package.  Simon Josefsson successfully uses a source
base of @file{gl/}, and a M4 base of @file{gl/m4/}, in several
packages.

After the @samp{--import} option on the command line comes the list of
Gnulib modules that you want to incorporate in your package.  The names
of the modules coincide with the filenames in Gnulib's @file{modules/}
directory.

Some Gnulib modules depend on other Gnulib modules.  @code{gnulib-tool}
will automatically add the needed modules as well; you need not list
them explicitly.  @code{gnulib-tool} will also memoize which dependent
modules it has added, so that when someday a dependency is dropped, the
implicitly added module is dropped as well (unless you have explicitly
requested that module).

If you want to cut a dependency, i.e. not add a module although one of
your requested modules depends on it, you may use the option
@samp{--avoid=@var{module}} to do so.  Multiple uses of this option are
possible.  Of course, you will then need to implement the same interface
as the removed module.

A few manual steps are required to finish the initial import.
@code{gnulib-tool} printed a summary of these steps.

First, you need to make sure Autoconf can find the macro definitions
in @file{gnulib-comp.m4}.  Use the @code{ACLOCAL_AMFLAGS} specifier in your
top-level @file{Makefile.am} file, as in:

@example
ACLOCAL_AMFLAGS = -I m4
@end example

You are now ready to call the M4 macros in @code{gnulib-comp.m4} from
@file{configure.ac}.  The macro @code{gl_EARLY} must be called as soon
as possible after verifying that the C compiler is working.
Typically, this is immediately after @code{AC_PROG_CC}, as in:

@example
...
AC_PROG_CC
gl_EARLY
...
@end example

The core part of the gnulib checks are done by the macro
@code{gl_INIT}.  Place it further down in the file, typically where
you normally check for header files or functions.  For example:

@example
...
# For gnulib.
gl_INIT
...
@end example

@code{gl_INIT} will in turn call the macros related with the
gnulib functions, be it specific gnulib macros, like @code{gl_FUNC_ALLOCA}
or autoconf or automake macros like @code{AC_FUNC_ALLOCA} or
@code{AM_FUNC_GETLINE}.  So there is no need to call those macros yourself
when you use the corresponding gnulib modules.

You must also make sure that the gnulib library is built.  Add the
@code{Makefile} in the gnulib source base directory to
@code{AC_CONFIG_FILES}, as in:

@example
AC_CONFIG_FILES(... lib/Makefile ...)
@end example

You must also make sure that @code{make} will recurse into the gnulib
directory.  To achieve this, add the gnulib source base directory to a
@code{SUBDIRS} Makefile.am statement, as in:

@example
SUBDIRS = lib
@end example

or if you, more likely, already have a few entries in @code{SUBDIRS},
you can add something like:

@example
SUBDIRS += lib
@end example

Finally, you have to add compiler and linker flags in the appropriate
source directories, so that you can make use of the gnulib library.
Since some modules (@samp{getopt}, for example) may copy files into
the build directory, @file{top_builddir/lib} is needed as well
as @file{top_srcdir/lib}.  For example:

@example
...
AM_CPPFLAGS = -I$(top_srcdir)/lib -I$(top_builddir)/lib
...
LIBADD = lib/libgnu.a
...
@end example

Don't forget to @code{#include} the various header files.  In this
example, you would need to make sure that @samp{#include "strdup.h"}
is evaluated when compiling all source code files, that want to make
use of @code{strdup}.

When an include file is provided by Gnulib
you shouldn't try to include the corresponding system header files
yourself, but let the gnulib header file do it.  The ordering
of the definition for some symbols may be significant; the Gnulib
header files take care of that.

For example, to use the @code{time_r} gnulib module you should
use include header file provided by the gnulib, and so
@samp{#include "time_r.h"}, but you shouldn't explicitly
@samp{#include <time.h>} as it is already done in @file{time_r.h}
before the redefinition of some symbols.

@node Modified imports
@section Modified imports

You can at any moment decide to use Gnulib differently than the last time.

If you only want to use more Gnulib modules, simply invoke
@command{gnulib-tool --import @var{new-modules}}.  @code{gnulib-tool}
remembers which modules were used last time.  The list of modules that
you pass after @samp{--import} is @emph{added} to the previous list of
modules.

For most changes, such as added or removed modules, or even different
choices of @samp{--lib}, @samp{--source-base} or @samp{--aux-dir}, there
are two ways to perform the change.

The standard way is to modify manually the file @file{gnulib-cache.m4}
in the M4 macros directory, then launch @samp{gnulib-tool --import}.

The other way is to call @command{gnulib-tool} again, with the changed
command-line options.  Note that this doesn't let you remove modules,
because as you just learned, the list of modules is always cumulated.
Also this way is often impractical, because you don't remember the way
you invoked @code{gnulib-tool} last time.

The only change for which this doesn't work is a change of the
@samp{--m4-base} directory.  Because, when you pass a different value of
@samp{--m4-base}, @code{gnulib-tool} will not find the previous
@file{gnulib-cache.m4} file any more... A possible solution is to manually
copy the @file{gnulib-cache.m4} into the new M4 macro directory.

In the @file{gnulib-cache.m4}, the macros have the following meaning:
@table @code
@item gl_MODULES
The argument is a space separated list of the requested modules, not including
dependencies.

@item gl_AVOID
The argument is a space separated list of modules that should not be used,
even if they occur as dependencies.  Corresponds to the @samp{--avoid}
command line argument.

@item gl_SOURCE_BASE
The argument is the relative pathname of the directory containing the gnulib
source files (mostly *.c and *.h files).  Corresponds to the
@samp{--source-base} command line argument.

@item gl_M4_BASE
The argument is the relative pathname of the directory containing the gnulib
M4 macros (*.m4 files).  Corresponds to the @samp{--m4-base} command line
argument.

@item gl_TESTS_BASE
The argument is the relative pathname of the directory containing the gnulib
unit test files.  Corresponds to the @samp{--tests-base} command line argument.

@item gl_LIB
The argument is the name of the library to be created.  Corresponds to the
@samp{--lib} command line argument.

@item gl_LGPL
The presence of this macro corresponds to the @samp{--lgpl} command line
argument.  It takes no arguments.

@item gl_LIBTOOL
The presence of this macro corresponds to the @samp{--libtool} command line
argument.  It takes no arguments.

@item gl_MACRO_PREFIX
The argument is the prefix to use for macros in the @file{gnulib-comp.m4}
file.  Corresponds to the @samp{--macro-prefix} command line argument.
@end table

@node Simple update
@section Simple update

When you want to update to a more recent version of Gnulib, without
changing the list of modules or other parameters, a simple call
does it:

@smallexample
$ gnulib-tool --import
@end smallexample

This will create, update or remove files, as needed.

@node CVS Issues
@section CVS Issues

All files created by @code{gnulib-tool}, except @file{gnulib-cache.m4},
should be treated like generated source files, like for example a
@file{parser.c} file is generated from @file{parser.y}.

In projects which commit all source files, whether generated or not, into
CVS, the @code{gnulib-tool} generated files should all be committed.

In projects which customarily omit from the CVS all files that generated
from other source files, all these files and directories would not be
added into CVS.  The only file that must be added to CVS is
@file{gnulib-cache.m4} in the M4 macros directory.  Also, the script for
restoring files not in CVS, customarily called @file{autogen.sh} or
@file{bootstrap.sh}, will typically contain the statement for restoring
the omitted files:

@smallexample
$ gnulib-tool --update
@end smallexample

The @samp{--update} option operates much like the @samp{--import} option,
but it does not offer the possibility to change the way Gnulib is used.
Also it does not report in the ChangeLogs the files that it had to add
because they were missing.

@node Copying This Manual
@appendix Copying This Manual

@menu
* GNU Free Documentation License::  License for copying this manual.
@end menu

@include fdl.texi


@node Index
@unnumbered Index

@printindex cp

@bye