- the final '\0'. When EOF is reached (i.e. on the call
- after the last token is read), -1 is returned and tokenbuffer
- isn't modified.
+ the final '\0'. Upon EOF (i.e. on the call after the last
+ token is read) or error, return -1 without modifying tokenbuffer.
+ The EOF and error conditions may be distinguished in the caller
+ by testing ferror (STREAM).