X-Git-Url: http://erislabs.net/gitweb/?a=blobdiff_plain;f=ckc302.txt;fp=ckc302.txt;h=13a661afc15c58b8e1de43e25e038e506442e6af;hb=d17b788660e379cc5381a5412e59e8a371c20586;hp=0000000000000000000000000000000000000000;hpb=078fa1ca577352acdb52f8e990c0e370a8841641;p=ckermit.git diff --git a/ckc302.txt b/ckc302.txt new file mode 100644 index 0000000..13a661a --- /dev/null +++ b/ckc302.txt @@ -0,0 +1,7881 @@ +C-KERMIT 9.0 CHANGE LOG (Changes since 8.0.207 / K95 2.1.3 January 2003) + + Chronological order. + Go to the bottom to find the newest edits. + + F. da Cruz, The Kermit Project, Columbia University, NYC. + Last update: 28 June 2011. + +FTP USER, FTP ACCOUNT, plus the various prompts and switches for FTP username, +password, and account all neglected to strip quotes, and in most cases quotes +are necessary to specify a username that contains spaces. ckcftp.c, +15 Jan 2003. + +FTP MPUT f1 f2 f3... gets a parse error if any of the fn's do not match an +existing file. This is bad for scripts. In doftpput(), cmfdb() looks for +keywords (switches) or CMIFI. When it hits CMIFI, it exits from the initial +parse loop and then does additional cmifi()s in a loop until done. The most +obvious fix is to parse each field with cmfdb(CMIFI,CMFLD), i.e. fall back to +CMFLD if CMIFI doesn't match anything. Then if CMFLD was used, we don't add +the filespec to the list. This is a rather big change but it seems to work. +No error messages or failures happen for non-matching fields, but an error +message is printed (and the MPUT command fails) if none of the fields match +any files. This fix got in too late for 2.1.3; workaround: use C-Shell +like wildcard list (ftp mput "{*.abc,foo.*}"). ckcftp.c, 16 Jan 2003. + +GREP did not pass its pattern through the expander, thus variables could +not be used for patterns. This must have been an oversight -- I can't find +anything in my notes about it. Fixed in dogrep(): ckuus6.c, 24 Jan 2003. + +New makefile target for HP-UX 11.xx with OpenSSL from Tapani Tarvainen. +makefile, 31 Jan 2003. + +From Jeff: + . Avoid core dump when dereferencing tnc_get_signature(): ckuus4.c. + . Bump version numbers to 8.0.208, 2.1.4: ckcmai.c. + +Added /NOLOGIN to FTP [OPEN]. ckcftp.c, 10 Feb 2003. + +Don't dump core if FTP DEBUG is ON and FTP OPEN does not include a service. +openftp(): ckcftp.c, 10 Feb 2003. + +HELP PATTERN text incorrectly identified commands and functions with +floating and anchored patterns. The corrected lists are: +Floating: GREP, TYPE /MATCH:, /EXCEPT: patterns, \farraylook(), +Anchored: IF MATCH, file-matching wildcards, \fsearch(), \frsearch() +ckuus2.c, 10 Feb 2003. + +INPUT n \fpattern(xxx) did not work for case-independent comparisons. +Fixed in doinput(): ckuus4.c, 10 Feb 2003. + +It seems \fpattern() didn't work with MINPUT at all. There was no code to +handle \fpattern() in the MINPUT parse loop, so it never worked. The code +had to be totally rewritten to use cmfld() in a loop, rather than cmtxt() +and then cksplit(). Furthermore, whenever any of the fields was an +\fjoin(), this had to be split. ckuusr.c, 10 Feb 2003. + +Macro replacement via \m() and \fdefinition() does not work as advertised +(i.e. case sensitively) for associative array elements; e.g. \m(xxx) is +treated the same as \m(xxx), contrary to section 7.10.10 of the C-Kermit +7.0 update notes, and to the fact that the two really do exist separately. +Fixed by adding a static function isaarray(s) which succeeds if s is an +associative array reference and fails otherwise, and then having \m() +and \fdef() call mxxlook() (case-sensitive lookup) if isaarray(), otherwise +(as before) mxlook()). ckuus4.c, 11 Feb 2003. + +Fixed FTP OPEN to allow the /USER switch to override SET FTP AUTOLOGIN OFF, +just as /NOLOGIN overrides SET FTP AUTOLOGIN ON. ckcftp.c, 11 Feb 2003. + +In K95, "set key \1234 \27H" (any SET KEY command in which the first char of +the definition was backslash, and the ONLY character after the backslash +quantity was an uppercase letter, that letter would be lowercased). Diagnosis: +xlookup() poking its argument (see notes from July 2000). Jeff sent a fix. +ckucmd.c, 15 Feb 2003. + +Ran my S-Expression torture test to make sure Sexps still worked. They do, +except the bitwise & and | operators were broken, e.g. (& 7 2) and (| 1 2 4) +get "Invalid operand" errors. Jeff's code had added an early failure return +from the lookup loop when when a single-byte keyword matched a keyword that +started with the same byte but was more than one byte long. So "&" would hit +"&&" and fail instead of continuing its search (xlookup tables aren't sorted +so there can be no early return). Fixed in xlookup(): ckucmd.c, 16 Feb 2003. + +Got rid of "krbmit" target from makefile. It's still there, but we don't +use it any more. All secure targets now use "xermit", and produce a binary +called wermit, just like the regular ones do (except the old ckucon.c ones). +Non-secure targets, since they don't define any of the security symbols, +wind up compiling and linking to (mostly) empty security modules. makefile, +15 Feb 2003. + +Added \fcvtdate(xxx,3) to format its result in MDTM format (yyyymmddhhmmss, +all numeric, no spaces or punctuation). Of course these numeric strings +are too big to be 32-bit numbers and are useless for arithmetic, but they're +useful for lexical comparison, etc. ckuus[24].c, 16 Feb 2003. + +The following FTP commands did not set FAILURE when they failed: RMDIR, +CD, CDUP, Fixed in the corresponding doftpblah() routines. ckcftp.c, +16 Feb 2003. + +RENAME would sometimes not print an error message when it failed, e.g. in K95 +when the destination file already existed. ckuus6.c, 17 Feb 2003. + +Fixed COPY error messages, which did not come out in standard format when +/LIST was not included. ckuus6.c, 17 Feb 2003. + +Fixed #ifdefs in ck_crp.c to allow nonsecure builds on old platforms like +System V/68 R3. 19 Feb 2003. + +Similar treatment for ck_ssl.c. 20 Feb 2003. + +From Jeff, 21 Feb 2003: + . AIX53 and AIX52 symbols for ckcdeb.h, makefile. + . New gcc targets for various AIX 4.x/5.x versions: makefile. + . Copyright date updates: ck_crp.c, ck_ssl.c. + . ENABLE/DISABLE QUERY broken because keyword table out of order: ckuusr.c. + . Fixed the use of HTTP proxies for HTTP [RE]OPEN for Unix: ckcnet.c. + +Also for K95 only: Allow file transfer when K95 is invoked on the remote end +of a connection to a Pragma Systems Terminal Server connection; automatically +SET EXIT HANGUP OFF when invoked with open port handle ("k95 -l nnnn"). + +"cd a*" failed even when "a*" matched only one directory. Fixed in cmifi(): +ckucmd.c, 21 Feb 2003. + +In the Unix version, replace "extern int errno;" with "#include " +if __GLIBC__ is defined, since glibc now defines a thread-specific errno. +ckcdeb.h, 26 Feb 2003. + +Added #ifdefs to skip compilation of ckuath.c in nonsecure builds. Tested +by building both secure and regular versions in Linux. ckuath.c, 26 Feb 2003. + +Ran the build-in-84-different-configurations script on Linux to make sure it +still builds with all different combinations of feature selection options. +All OK. 26 Feb 2003. + +Built on VMS. Needed to add a prototype for mxxlook*() to ckuusr.h; built +OK otherwise. 26 Feb 2003. + +From Jeff: More #ifdef shuffling for nonsecure builds: ckuath.c, ck_ssl.c, +27 Feb 2003. + +Added code to ensure \v(download) ends in a directory separator in Unix, +Windows, and OS/2. ckuus7.c, 27 Feb 2003. + +Added code to K95 zfnqfp() to tack on directory separator when returning +a directory name. ckofio.c, 27 Feb 2003. + +Somehow an old copy of ckuath.c popped to replace the new one. Put the new +one back. 28 Feb 2003. + +From Jeff: Fix typo in my K95 zfnqfp() code from yesterday; fixes for handling +UNCs uniformly, no matter which way their slashes are leaning. ckofio.c, +28 Feb 2003. + +At Jeff Mezei's suggestion, separate text and binary mode open sequences +for VMS session log. ckvfio.c, 28 Feb 2003. + +Added freebsd48 target for FreeBSD 4.8. makefile, 1 Mar 2003. + +Changed Mac OS X entries to include -DUSE_STRERROR. makefile, 2 Mar 2003. + +Fixed GETOK /GUI to evaluate its text argument. ckuus6.c, 3 Mar 2003. + +Jeff fixed the K95 Dialer QUICK dialog to (a) allow templates, and (b) have +a Save-As option. 3 Mar 2003. + +Jeff fixed a problem with the Xmodem-CRC checksum being crunched whenever +there was a retransmission. 7 Mar 2003. + +Added target/banner for Tru64 5.1B. makefile, ckuver.h, 5 Mar 2003. + +In Unix, the zcopy() routine (used by the COPY command) reset the user's umask +to 0 for the remainder of the Kermit process lifetime. The bug was in +ckufio.c 8.0.194, 24 Oct 2002, and is fixed in ckufio.c 8.0.195, 6 Mar 2003. +Of course this happened after building 155 C-Kermit 8.0.208 binaries. (But +before officially releasing 8.0.208.) + +In the VMS version, changed: + + while ((n--) && xx_inc(2) > -1) ; +to: + while ((n--) && xx_inc(2) >= 0) ; + +to suppress the "...is being compared with a relational operator to a constant +whose value is not greater than zero" warning. ckvtio.c, 7 Mar 2002. + +Added a debug call to dologend in hopes of catching overzealous Locus +switching, which seems to happen only in K95. ckuus3.c, 7 Mar 2002. + +Rebuilt binaries for some of the more current Unix releases: AIX 4.3.3-5.1, +Solaris 7-9 , Red Hat 7.0-8.0, Slackware 8.1, Freebsd 4.7-4.8, NetBSD 1.6, +OpenBSD 3.2, Unixware 7.1.3, Open Unix 8, OSR5.0.6a, etc. A Unix binary with +COPY umask fix shows a 6 Mar 2003 date for "UNIX File support" in SHOW +VERSIONS; a binary without the fix shows 24 Oct 2002. + +C-Kermit 8.0.208 dated 14 March 2003 released on 10 March 2003. + +---8.0.208--- + +From Jeff 13 Mar 2003: + . Updated SSL module allows importation of tickets from host. + . freebsd50+openssl target: makefile. + . FTP PUT /PERMISSIONS error message for K95: ckcftp.c. + +Fixed MINPUT to strip quotes or braces from around targets (this was broken +on Feb 10th). Thanks to Jason Heskett for discovering and reporting this +(killer) bug. ckuusr.c, 14 Mar 2003. + +Changed version number to 209 Dev.00. ckcmai.c, 14 Mar 2003. + +While debugging the alphapage script, I found that the command "minput 8 \6\13 +\21\13 \13\27\4\13 \30\13" gets "?Not confirmed" in 8.0.208 and 8.0.209, but +not in 206 and earlier. This problem too was introduced on Feb 10th by +changing MINPUT parsing from cmtxt() followed by cksplit() to cmfld() in a +loop. cmfld() uses setatm() to return its result and of course setatm() +breaks on \13. Changing setatm() not to do this would break everything else. +But cmfld() has no arguments that let us tell it to do anything different in +this case. Changing the API would be a disaster. The only solution is to add +an "MINPUT ACTIVE" (minputactive) global variable that tells cmfld() to tell +setatm() not to break on CR. Now MINPUT with braced targets containing CR +and/or LF works in 209, 206, and 201 (but not 208). ckucmd.c, ckuusr.c, +ckuus5.c, 15 Mar 2003. + +MINPUT n \fjoin(&a) works OK if all the members of \&a[] are text strings, but +if they are strings of control chars (as above), they don't get separated by +the spaces. For example in: + + dcl \&a[] = "\4\5" "\6\7" xxx + minput 10 \fjoin(&a) + +MINPUT gets two targets: "aaa" and "\4\5 \6\7 xxx". The bug was in the +cksplit() call in the \fjoin() case of MINPUT: it needed to specify an +include set consisting of all the control characters except NUL. ckuusr.c, +16 Mar 2003. + +But there's still a problem: + + dcl \&a[] = "\4\5\13\10" "\6\7" "xxx" + +creates an array whose first member is "^D^E (one doublequote included). But +if braces are used instead, there's no problem. Same deal as MINPUT: cmfld() +breaks on CR or LF, thus the end quote is lost. If I set minputactive for +DECLARE initializers too, that fixes it. Is there any reason not to do this? +Can't think of any (famous last words)... ckuusr.c, 16 Mar 2003. + +Since it has multiple applications, changed the flag's name from minputactive +to keepallchars. ckucmd.c, ckuus[r5].c, 16 Mar 2003. + +\v(exedir) wasn't being set correctly (it included the program name as well +as the directory). Fixed in getexedir(): ckuus4.c, 16 Mar 2003. + +SET CARRIER-WATCH "auto matic" (spurious space in supplied keyword). +Cosmetic only; it still worked. Fixed in setdcd(): ckuus3.c, 16 Mar 2003. + +"directory a b c" listed too many files -- all files whose names END WITH a, +b, or c, rather than the files whose names WERE a, b, or c. Diagnosis: The +filespec is changed into a pattern: {a,b,c}, which is the correct form. It is +passed to nzxpand(), which goes through the directory getting filenames and +sending each one to ckmatch() with the given pattern. ckmatch() receives the +correct pattern but then prepends a "*" -- that's not right. It's not just +in filename matching either. The following succeeds when it shouldn't: + + if match xxxxc {{a,b,c}} + +Changing ckmatch() to not prepend the "*" to each segment fixes the command +above but breaks lots of others. Running through the "match" torture-test +script shows the problem occurs only when the {a,b,c} list is the entire +pattern, and not embedded within a larger pattern. Testing for this case +fixed the problem. ckmatch(): ckclib.c, 16 Mar 2003. + +Fixed FTP MODTIME to not print anything if QUIET ON. ckcftp.c, 16 Mar 2003. + +Picked up a new ckuath.c from Jeff, not sure what the changes are. 16 Mar 2003. + +Did a few regular and secure builds to make sure I didn't wreck anything. + +Changed version number to 209 (final). ckcmai.c, 16 Mar 2003. + +Jason Heskett found another bug: if you define a macro FOO inside the +definition of another macro BAR, and FOO's definition includes an odd number +of doublequotes (such as 1), FOO's definition absorbs the rest of BAR's +definition. Example: + + def TEST { + .foo = {X"} + sho mac foo + } + do test + sho mac foo + +Results in: + + foo = {X"}, sho mac foo + +Diagnosis: the TEST definition becomes: + + def TEST .foo = {X"}, sho mac foo + +and the macro reader is erroneously treating the doublequote as an open +quote, and then automatically closes the quote at the end of the definition. +The error is that a doublequote should be significant only at the beginning of +a field. But the macro reader isn't a command parser; it doesn't know what +a field is -- it's just looking for commas and skipping over quoted ones. +First we have to fix an oversight: SET COMMAND DOUBLEQUOTING OFF should have +worked here, but it wasn't tested in this case. Fixed in getncm(): ckuus5.c, +17 Mar 2003. + +There are only certain cases where it makes sense to treat doublequotes as +significant: + + . An open quote must be at the beginning or preceded by a space. + . A close quote is only at the end or else followed by a space. + +This too was fixed in getncm(): ckuus5.c, 17 Mar 2003. + +A fix from Jeff SSL/TLS FTP data decoding. ckcftp.c, 18 Mar 2003. + +Tried building C-Kermit on a Cray Y-MP with UNICOS 9.0. "int suspend", +declared in ckcmai.c and used in many modules, conflicts with: + + unistd.h:extern int suspend __((int _Category, int _Id)); + +The "=Dsuspend=xsuspend" trick doesn't work for this; there is no way around +the conflict other than to rename the variable: ckcmai.c, ckutio.c, +ckuus[35xy].c. 26 Mar 2003. VMS and K95 not affected. + +OK that gets us past ckcmai.c... Then in ckutio.c I had to add a new #ifdef +around the LFDEVNO setting, because the Cray didn't have mkdev.h. Could not +find a Cray-specific manifest symbol, so I made a new makefile target (cray9) +that sets this symbol. Having done this I have no idea what kind of lockfile +would be created, but I also doubt if anybody dials out from a Cray. The +binary should run a C90, J90, or Y-MP. makefile, 26 Mar 2003. + +Added a target for SCO OSR5.0.7. makefile, ckuver.h, 30 Mar 2003. + +Changed since 208: +makefile ckuver.h ckcmai.c ckclib.c ckcftp.c ckucmd.c ckuus*.c ckutio.c. + +---8.0.209--- + +From Mark Sapiro, a fix for the March 17th doublequote fix, getncm(): ckuus5.c, +4 Apr 2003. + +From Jeff, 29 Apr 2003: + . Corrected target for HP-UX 11.00 + OpenSSL: makefile, + . Do not allow WILL AUTH before WONT START_TLS: ckctel.h ckctel.c + . Add hooks for SFTP and SET/SHOW SFTP: ckcdeb.h ckuusr.h ckuusr.c ckuus3.c + . Add SKERMIT ckuusr.h ckuusr.c + . Add ADM-5 terminal emulation: ckuus7.c, ckuus5.c + . Uncomment and update HELP SET SSH V2 AUTO-REKEY: ckuus2.c + . Enable IF TERMINAL-MACRO and IF STARTED-FROM-DIALER for C-Kermit: ckuus6.c + . Fix conflicting NOSCROLL keyword definition: ckuusr.h + . Set ttname when I_AM_SSH: ckuusy.c + . Add extended arg parsing for SSH, Rlogin, Telnet: ckuusy.c, ckuus4.c + . Security updates: ckuath.c, ck_ssl.c + . Change K95 version number to 2.2.0: ckcmai.c + . Save K95 term i/o state before executing keyboard macro: ckuus4.c + . Add tests for SSH Subsystem active during INPUT/OUTPUT/CONNECT: ckuus[45].c + . Enable K95 SET SSH V2 AUTO-REKEY: ckuus3.c + +SFTP and SET SFTP subcommands are implemented up to the case statements. + +Files of mine that Jeff hadn't picked up: + ckuver.h ckcftp.c ckutio.c ckuusx.c (just minor changes for last build-all) + +On 4 Jan 2003, SET RECEIVE MOVE-TO was changed to convert is argument to an +absolute path, which made it impossible to specify a relative path, then +move to different directories and have it apply relatively to each directory. +Changed this as follows: + + . Parser uses cmtxt() rather than cmdir() so it won't fail at parse time. + . If path is absolute, we fail at parse time if directory doesn't exist. + . In reof() we run the the path through xxstring (again, in case deferred + evaluation of variables is desired) and then, if not null, use it. + . If the directory doesn't exist, rename() fails and reof() returns -4, + resulting in a protocol error (this is not a change). We do NOT create + the directory on the fly. + +I also fixed SET SEND/RECEIVE RENAME-TO to parse with cmtxt() rather than +cmdir(), since it's parsing a text template, not a directory name, e.g. +"set receive rename-to file-\v(time)-v(date)-\v(pid)". This was totally +broken, since when I don't know. We don't call xxstring() in this parse, so +evaluation is always deferred -- I'd better not change this. ckuus7.c, +ckcfns.c, 1 May 2003. + +From Jeff, Sat May 3 14:15:23 2003: + . Pick up the right isascii definition for K95: ckctel.c + . malloc... ckuath.c (new safe malloc routines for K95) + . Add author listing: ckuus5.c + . SSH Heartbeat support (K95 only): ckuus[23].c + . Prescan --height and --width to avoid window resizing at startup: ckuusy.c + . Add checks for fatal() or doexit() called from sysinit(): ckuusx.c + . Move some K95-specific definitions to ckoker.h: ckcdeb.h + . Add support for ON_CD macro in zchdir(): ckufio.c + . Add a command to let FTP client authenticate with SSLv2: ckcftp.c + . Fix parsing of FTP file facts like "UNIX.mode": ckcftp.c + +ON_CD will need some explaining (to be done). It's implemented for Unix, +VMS, WIndows, and OS/2. + +The FTP file facts fix came from first exposure to the new OpenBSD FTP +server: ftp://ftp7.usa.openbsd.org/pub/os/OpenBSD/3.3/i386/ +The period in "UNIX.mode" caused an erroneous word break, adding junk to +the filename. + +About the malloc changes, Jeff says "K95 is not behaving well in low memory +environments. I'm not sure that C-Kermit does much better. The program does +not crash but it certainly does not behave the way the user expects it to. +I'm beginning to think that any malloc() error should be treated as fatal." + +Not visible in these changes because it's in K95-specific modules: Jeff made +SET ATTRIBUTES OFF and SET ATTRIBUTES DATE OFF apply to XYZMODEM transfers. + +From Jeff, 11 May 2003: + . Add support for SSH Keepalive to relevant SET command (K95): ckuus3.c + . Reduce max overlapped i/o requests from 30 to 7 (K95): ckuus7.c + . Don't call sysinit() in fatal(): ckuusx.c. + . Some new conditionalizations for SSL module: ck_ssl.c + +The doublequote-parsing fixes from March and April broke the SWITCH statement, +which is implemented by internally defining, then executing, a macro. If I +drop back to the old dumb handling of doublequotes, everything is fixed except +the problem of March 17th. But can we really expect getncm() to pre-guess +what the parser is going to do? getncm()'s only job is to find command +boundaries, which are represented by commas. Commas, however, is needed IN +commands too. We take a comma literally if it is quoted with \, or is inside +a matched pair of braces, parens, or doublequotes. It is not unreasonable to +require a doublequote in a macro definition to be prefixed by \ when it is to +be taken literally. The proper response to Jason Heskett's complaint of March +17th should have been to leave the code alone and recommand an appropriate +form of quoting: + + def TEST { + .foo = {X\"} + sho mac foo + } + +And this is what I have done. Another reason for sticking with the old method +is that it's explainable. The "improved" method, even if it worked, would be +be impossible to explain. Btw, in testing this I noticed that the switch-test +script made 8.0.201 dump core. Today's version is fine. The problem with +quoted strings inside of IF {...} clauses and FOR and WHILE loops is fixed +too. Perhaps "unbroken" would be a better word. ckuus5.c, 11 May 2003. + +Vace discovered that FTP MGET /EXCEPT:{... (with an unterminated /EXCEPT list) +could crash Kermit. Fixed in ckcftp.c, 11 May 2003. + +CONTINUE should not affect SUCCESS/FAILURE status. ckuusr.c, 11 May 2003. + +Fixed an oversight that goes back 15 years. While \{123} is allowed for +decimal codes, \x{12} and \o{123} were never handled. ckucmd.c, 11 May 2003. + +Added support for Red Hat and /usr/sbin/lockdev. Supposedly this +allows Kermit to be installed without setuid or setgid bits and still be able +to lock and use the serial device. Compiles and starts, but not tested. +ckcdeb.h, makefile, ckutio.c, ckuus5.c, 16 May 2003. + +From Jeff: FTP ASCII send data to host when FTP /SSL was in use was broken. +ftp_dpl is set to Clear when FTP /SSL is in use. This was causing the data to +be written to the socket with send() instead of the OpenSSL routines. +ckcftp.c, ckuath.c, 21 May 2003. + +From Jeff: Stuff for Kerberos 524: ckcdeb.h. Fixes for FTP; "FTP ASCII send +data did not properly compute the end of line translations. On Unix (and +similar platforms) the end of line was correct for no character sets but +incorrect when character sets were specified. On Windows/OS2, the end of line +was correct when character sets were specified and incorrect when they were +not. On MAC, both were broken. Also, FTP Send Byte counts were incorrect +when character sets were specified." ckcftp.c. 17 Jun 2003. + +From Jeff: fixes to HTTP /AGENT: and /USER: switch action: ckcnet.c ckuus3.c +ck_crp.c ckcftp.c ckuus2.c ckuusy.c ckuusr.c ckcnet.h, 21 Jun 2003. + +From Jeff: Fix SET DIALER BACKSPACE so it can override a previous SET KEY +(e.g. from INI file): ckuus7.c. Some SSL/TLS updates: ck_ssl.c. HTTP support +for VMS and other VMS improvements (e.g. a way to not have to hardwire the +C-Kermit version number into the build script) from Martin Vorlaender: +ckcnet.h, ckuus[r3].c, ckcdeb.h, ckvtio.c, ckcnet.c, ckvker.com. Built on +Solaris (gcc/ansi) and SunOS (cc/k&r). The new VMS script tests the VMS +version and includes HTTP support only for VMS 6.2 or later. 2 Jul 2003. + +Tried to build on our last VMS system but it seems to be dead. Looks like a +head crash (makes really loud noises, boot says DKA0 not recognized) (fooey, I +just paid good money to renew the VMS license). Tried building at another +site with: + + Process Software MultiNet V4.3 Rev A-X, + Compaq AlphaServer ES40, OpenVMS AXP V7.3 + Compaq C V6.4-008 on OpenVMS Alpha V7.3 + +Had to make a few corrections to ckvker.com. But still, compilation of +ckcnet.c bombs, indicating that the SELECT definition somehow got lost +somewhere since the 209 release (i.e. no SELECT type is defined so it falls +thru to "SELECT is required for this code"). But I don't see anything in +ckcdeb.h or ckcnet.[ch] that would explain this. Not ckvker.com either +(putting the old one back gives the same result). OK, I give up, maybe it's +just that I haven't tried building it on MultiNet recently. What about UCX? +Aha, builds fine there except for warnings about mlook, dodo, and parser in +ckvfio.c (because of ON_CD) -- I suppose I have #include ... (done) +Anyhow it builds OK and the HTTP code is active and almost works (HTTP OPEN +works; HTTP GET seems to succeed but creates an empty file every time). Tried +building under MultiNet at another installation; same bad result. + +OK so why won't it build for MultiNet? Comparing ckcnet.c with the 209 +version, not a single #ifdef or #include is changed. Tried building with +p3="NOHTTP" -- builds OK, aha. Where's the problem? Not ckcnet.h... +Not ckcdeb.h... OK I give up, will revisit this next time I get time to +do anything with the code. + +Later Jeff said "Martin did not implement VMS networking for the HTTP code. +All he did was activate the #define HTTP which happens to work because his +connections are using SSL/TLS connections. http_inc(), http_tol(), etc have +no support for VMS networking regardless of whether it is UCX or MULTINET. +The vast majority of HTTP connections are not secured by SSL/TLS. It makes no +sense to support HTTP on VMS until someone is willing to either do the work or +pay have the work done to implement VMS networking in that code base." So the +fix is to not enable HTTP for VMS after all. Removed the CKHTTP definition +for VMS from ckcdeb.h, 6 Jul 2003. + +Fixed ckvfio.c to #include (instead of ) to pick up +missing prototypes. 6 Jul 2003. + +From Arthur Marsh: solaris2xg+openssl+zlib+srp+pam+shadow and the corresponding +Solaris 7 target. makefile, 6 Jul 2003. + +Remove duplicate #includes for , , and from +ckcftp.c. 6 Jul 2003. + +Add -DUSE_MEMCPY to Motorola SV/68 targets because of shuffled #includes in +ckcftp.c. 8 Jul 2003. + +From Jeff: Fix problems mixing SSL and SRP without Kerberos. Plus a few minor +#define comment changes and a reshuffling of #defines in ckcdeb.h to allow me +to build on X86 Windows without Kerberos. ckcdeb.h, ck_crp.c, ckuath.c, +10 Jul 2003. + +From Jeff: updated ckuat2.h and ckuath.c, 29 Jul 2003. + +Mats Peterson noticed that a very small Latin-1 file would be incorrectly +identified as UCS-2 by scanfile(). Fixed in ckuusx.c, 29 Jul 2003. + +Fixed ACCESS macro definition to account for the fact that FIND is now a +built-in command. ckermit.ini, 30 Jul 2003. + +From Jeff: Fix for typo in urlparse() (svc/hos): ckuusy.c, 18 Aug 2003. + +From Jeff: Redhat9 makefile targets (needed for for OpenSSL 0.9.7): +makefile, 19 Aug 2003. + +GREP /NOLIST and /COUNT did too much magic, with some undesirable fallout: +"GREP /NOLIST /COUNT:x args" printed "file:count" for each file. "GREP +/COUNT:x /NOLIST args" did not print "file:count", but neither did it set the +count variable. Removed the magic. Also one of the GREP switches, +/LINENUMBERS, was out of order. Fixed in ckuus6.c, 20 Aug 2003. + +From Jeff: "Reorganizing code to enable building with different subsets of +options; a few typos corrected as well." ckcdeb.h, ckuver.h (for RH9), +ckcnet.c, ckuus7.c, ckuus3.c: 24 Aug 2003. + +Scanfile misidentified a big PDF file as text because the first 800K of it +*was* text (most other PDF files were correctly tagged as binary). Fixed +by adding a check for the PDF signature at the beginning of the file. +scanfile(): ckuusx.c, 25 Aug 2003. + +Ditto for PostScript files, but conservatively. Signature at beginning of +file must begin with "%!PS-Ado". If it's just "%!" (or something nonstandard +like "%%Creator: Windows PSCRIPT") we do a regular scan. Also added "*.ps" +to all binary filename patterns. ckuusx.c, 4 Sep 2003. + +Ditto (but within #ifndef NOPCLSCAN) for PCL (E) and PJL (%) files, +but no binpatterns (note: ".PCL" is the extension for TOPS-20 EXEC scripts). +ckuusx.c, 4 Sep 2003. + +Added comments about OpenSSL 0.9.7 to all linux+openssl targets. +makefile, 4 Sep 2003. + +From Jeff: Added - #define ALLOW_KRB_3DES_ENCRYPT. When this symbol is defined +at compilation Kermit will allow non-DES session keys to be used during Telnet +Auth. These session keys can then be used for Telnet Encrypt. The reason +this is not compiled on by default is that the MIT Kerberos Telnet does not +follow the RFC for constructing keys for ENCRYPT DES when the keys are longer +than 8 bytes in length. ckuath.c, ckuus5.c, 4 Sep 2003. + +"ftp mget a b c" succeeded if one or more of the files did not exist, even +with "set ftp error-action proceed". This is because the server's NLST file +list does not include any files that don't exist, so the client never even +tries to get them. Fortunately, the way the code is structured, this one was +easy to fix. ckcftp.c, 14 Sep 2003. + +From Jeff: Corrected code in ckcnet.c to ensure that Reverse DNS Lookups are +not performed if tcp_rdns is OFF. Fixed ck_krb5_getrealm() to actually return +the realm of the credentials cache and not the default realm specified in the +krb5.conf file. Previously krb5_cc_get_principal() was not being called. +Fixed ck_krb5_is_tgt_valid() to test the TGT in the current ccache and not the +TGT constructed from the default realm. ckcnet.c, ckuath.c, 14 Sep 2003. + +Marco Bernardi noticed that IF DIRECTORY could produce a false positive if +the argument directory had previously been referenced but then removed. This +is because of the clever isdir() cache that was added to speed up recursion +through big directory trees. Changed IF DIRECTORY to make a second check +(definitive but more expensive) if isdir() succeeds, and changed the +directory-deleting routine, ckmkdir(), to flush the directory cache (UNIX +only -- this also should be done in K95 but it's not critical). This was +done by adding a routine, clrdircache() to ckufio.c, which sets prevstat +to -1 and prevpath[0] to NUL. ckcfn3.c, ckuus6.c, ckufio.c, 18 Sep 2003. + +Marco reported the second fix still didn't work for him (even though it did +for me). Rather than try to figure out why, I concluded that the directory +cache is just not safe: a directory found a second ago might have been deleted +or renamed not only by Kermit but by some other process. Why did I add this +in the first place? The log says: + + Some debug logs showed that isdir() is often called twice in a row on the + same file. Rather than try to sort out clients, I added a 1-element cache + to Unix isdir(). ckufio.c, 24 Apr 2000. + +Experimentation with DIR and DIR /RECURSIVE does not show this happening at +all. So I #ifdef'd out the directory cache (see #ifdef ISDIRCACHE in ckufio.c; +ISDIRCACHE is not defined) and backed off the previous changes: ckufio.c, +ckcfn3.c, ckuus6.c, 28 Sep 2003. + +From Jeff: Replace the compile time ALLOW_KRB_3DES_ENCRYPT with a run-time +command SET TELNET BUG AUTH-KRB5-DES which defaults to ON: ckctel.[ch], +ckuus[234].c, ck_crp.c, ckuath.c. 4 Oct 2003. + +Allow DIAL RETRIES to be any positive number, and catch negative ones. +Also added code to check for atoi() errors (e.g. truncation). At least on +some platforms (e.g. Solaris) atoi() is supposed to set errno, but it +doesn't. ckuus3.c, ckucmd.c, 4 Oct 2003. + +Added /DEFAULT: to ASK-class commands (ASK, ASKQ, GETOK): + + . For popups: no way to send defaults to popup_readtext() or popup_readpass(). + . For GUI ASK[Q], pass default to gui_txt_dialog(). + . For GUI GETOK, convert "yes" "ok" or "no" default to number for uq_ok(). + . For Text GETOK, add default to cmkey(). + . For Text ASK[Q], add default to cmtxt(). + . For GETC, GETKEY, and READ: no changes. + +GETOK, ASK, and ASKQ with /TIMEOUT: no longer fail when the timer goes off +if a /DEFAULT was supplied. The GUI functions (uq_blah) don't seem to +support timeouts. Only the text version has been tested. ckuus[26].c, +4 Oct 2003. + +From Jeff: add /DEFAULT: for popups. ckuus6.c. 6 Oct 2003. + +Change SET DIAL INTERVAL to be like SET DIAL RETRIES. ckuus[34].c, 6 Oct 2003. + +Added target for HP-UX 10/11 + OpenSSL built with gcc, from Chris Cheney. +Makefile, 12 Oct 2003. + +From Jeff, 6 Nov 2003: + . #ifdef adjustments: ckcftp.c, ckcdeb.h + . Fix spurious consumption of first byte(s) on Telnet connection: ckctel.c + . Another HP PJL test for scanfile: ckuusx.c. + . K95: Recognize DG4xx protected fields in DG2xx emulation: ckuus7.c. + . Add SSLeay version display to SHOW AUTH command: ckuus7.c + . Improved SET MOUSE CLEAR help text: ckuus2.c. + . Improved Kverbs help text: ckuus2.c (+ new IBM-3151 Kverbs). + . Some changes to ck_ssl.c, ckuath.c. + +From PeterE, 10 Nov 2003: + . Improved HP-UX 10/11 makefile targets for OpenSSL. + . #ifdef fix for OpenSSL on HP-UX: ck_ssl.c. + +Another new makefile from PeterE with improved and integrated HP-UX targets. +12 Nov 2003. + +A couple fixes to the solaris9g+krb5+krb4+openssl+shadow+pam+zlib target +from Jeff. Added a solaris9g+openssl+shadow+pam+zlib target. makefile, +21 Nov 2003. + +From Jeff, 30 Nov 2003: + . Fix SEND /MOVE-TO: ckuusr.c. + . Fix K95 SET TITLE to allow quotes/braces around text: ckuus7.c. + . Improved "set term autodownload ?" response: ckuus5.c. + . Fix SHOW FEATURES to specify the protocol for encryption: ckuus5.c + . Make {SEND, RECEIVE} {MOVE-TO, RENAME-TO} work for XYZMODEM (K95 only). + +From Jeff: 7 Jan 2004: + . At one point Frank started to add a timer parameter to the + uq_txt() function but he only did it for the non-ANSI + compilers. I added it for the ANSI compilers, fixed the + prototypes and provided a default value easily changed + DEFAULT_UQ_TIMEOUT: ckcker.h, ckuus[36].c, ck_ssl.c, ckcftp.c, ckuath.c. + . Fixed SET TERMINAL DEBUG ON (typo in variable name): ckuus7.c. + . Fixed BEEP INFORMATION; previously it made no sound, now uses + MB_ICONQUESTION. ckuusx.c. + +From Ian Beckwith (Debianization), 7 Jan 2004: + . Search dir/ckermit for docs, as well as dir/kermit in cmdini(): ckuus5.c. + . New linux+krb5+krb4+openssl+shadow+pam target (kitchen sink minus SRP, + which Debian does not distribute): makefile. + ? Mangles the DESTDIR support in makefile to install into a staging area: + makefile (I didn't take this one yet). + +Updated copyright notices for 2004, all modules. 7 Jan 2004. + +Added INPUT /NOMATCH, allowing INPUT to be used for a fixed amount of time +without attempting to match any text or patterns, so it's no longer +necessary to "input 600 STRING_THAT_WILL_NEVER_COME". If /NOMATCH is +included, INPUT succeeds if the timeout expires, with \v(instatus) = 1 +(meaning "timed out"); fails upon interruption or i/o error. ckuusr.h, +ckuus[r24].c, 7 Jan 2004. + +Added SET INPUT SCALE-FACTOR . This scales all INPUT timeouts by the +given factor, allowing time-sensitive scripts to be adjusted to changing +conditions such as congested networks or different-speed modems without +having to change each INPUT-class command. This affects only those timeouts +that are given in seconds, not as wall-clock times. Although the scale +factor can have a fractional part, the INPUT timeout is still an integer. +Added this to SHOW INPUT, and added a \v(inscale) variable for it. +ckuusr.h, ckuus[r257].c, 7 Jan 2004. + +undef \%a, \fverify(abc,\%a) returns 0, which makes it look as if \%a is a +string composed of a's, b's, and/or c's, when in fact it contains nothing. +Changed \fverify() to return -1 in this case. ckuus4.c, 12 Jan 2004. + +\fcode(xxx) returned an empty string if its argument string was empty. This +makes it unsafe to use in arithmetic or boolean expressions. Changed it to +return 0 if its argument was missing, null, or empty. ckuus4.c, 12 Jan 2004. + +Updated \verify() and \fcode() help text. ckuus2.c, 12 Jan 2004. + +While setting up IKSD, Ian Beckwith noticed that including the --initfile: +option caused Kermit to start parsing its own Copyright string as if it were +the command line, and eventually crash. I couldn't reproduce on Solaris / +Sparc but I could in Linux / i386 (what Ian is using) -- a change from Jeff +on 28 Apr 2003 set the command-line arg pointer to a literal empty string in +prescan() about line 1740 of of ckuus4.c; the pointer is incremented next +time thru the loop, resulting in random memory being referenced. Fixed by +setting the pointer to NULL instead of "". ckuus4.c, 12 Jan 2004. + +declare \&a[999999999999999] would dump core on some platforms. atoi() +or whatever would truncate the dimension to maxint. When we add 1 to the +result, we get a negative number, which is used as an index, loop test, etc. +Fixed both dodcl() and dclarray() to check for (n+1 < 0). ckuus[r5].c, +12 Jan 2004. + +Unix zchki() would fail on /dev/tty, which is unreasonable. This prevented +FOPEN /READ from reading from the terminal. zchki() already allowed for +/dev/null, so I added /dev/tty to the list of specials. Ditto for FOPEN +/WRITE and zchko(). ckufio.c 13 Jan 2004. + +Added untabify() routine to ckclib.[ch], 13 Jan 2004. +Added FREAD /TRIM and /UNTABIFY. ckuus[27].c, 13 Jan 2004. +Added \funtabify(). ckuusr.h, ckuus[24].c, 13 Jan 2004. + +Dat Nguyen noticed that (setq u 'p') followed by (u) dumped core. This was +caused by an over-clever optimization that skipped mallocs for short +literals, but then went on later to try to free one that hadn't been +malloc'd. Fixed in dosexp(): ckuus3.c, 14 Jan 2004. + +Catch another copyright date. ckuus5.c, 14 Jan 2004. + +Fixed SWITCH to work even when SET COMMAND DOUBLEQUOTE OFF (from Mark +Sapiro). ckuus5.c, 15 Jan 2004. + +Changed version to 8.0.211 so scripts can test for recently added features. +ckcmai.c, 15 Jan 2004. + +Fixed a glitch in K95 "help set port". ckuus2.c, 20 Jan 2004. + +Fix from Jeff: Connections to a TLS-aware protocol which require a reconnect +upon certificate verification failure could not reconnect if the connection +was initiated from the command line or via a URL. ckctel.c ckcmai.c +ckuusr.c ckuus7.c ckuusy.c, 20 Jan 2004. + +From Alex Lewin: makefile target and #ifdef for Mac OS X 10.3 (Panther): +makefile, ckcnet.c, 7 Feb 2004. + +Added KFLAGS to sco32v507 targets to make PTY and SSH commands work. The +same flags could probably also be added to earlier OSR5 targets but they +have not been tested there. makefile, 7 Feb 2004. + +Checked a complaint that "LOCAL &a" did not make array \&a[] local. Indeed +it did not, and can not. You have to use the full syntax in the LOCAL +command, "LOCAL \&a[]", or else it doesn't know it's not a macro named &a. +7 Feb 2004. + +Fixed some confusion in creating IKSD database file and temp-file names. +I was calling zfnqfp() without remembering that the path member of the +returned struct included the filename, so to get just the directory name, +I needed to strip the filename from the right. ckuusy.c, 2 Mar 2004. + +New ckuath.c, ck_ssl.c from Jeff. 2 Mar 2004. + +Updated Jeff's affiliation in VERSION command text. ckuusr.c, 2 Mar 2004. + +Designation changed from Dev.00 to Beta.01. ckcmai.c, 2 Mar 2004. + +Fixed zrename() syslogging -- it had success and failure reversed. +Beta.02: ckufio.c, 4 Mar 2004. + +Problem: when accessing IKSD via a kermit:// or iksd:// URL, and a user ID +is given but no password, doxarg() set the password to "" instead of leaving +it NULL, but all the tests in dourl() are for NULL. Fixed in doxarg(): +ckuusy.c, 5 Mar 2004. + +The logic in dourl() about which macro to construct (login and connect, +login and get directory listing, or login and fetch a file) was a bit off, +so all three cases were not handled. ckcmai.c, 5 Mar 2004. + +Trial Beta builds: + . HP-UX B.11.11 PA-RISC + . HP-UX B.11.23 IA64 + . Tru64 4.0G Alpha + . Tru64 5.1B Alpha + . Debian 3.0 i386 + . Red Hat ES 2.1 i386 + . Slackware 9.1 i386 + . VMS 7.3-1 Alpha + UCX 5.3 + . VMS 7.3-1 Alpha no TCP/IP + . VMS 7.3 Alpha MultiNet 4.3 A-X + . SCO UnixWare 7.1.4 i386 + . SCO OSR5.0.7 i386 + . Solaris 9 Sparc + +Fixed compiler warning in doxarg() caused by typo (NULL instead of NUL) in +the 5 March doxarg() edit. ckuusy.c, 9 Mar 2004. + +IKSD (kermit://) command-line URLs did not work right if the client had +already preauthenticated with Kerberos or somesuch because they tried to log +in again with REMOTE LOGIN. The macros constructed in doxarg() needed to +check \v(authstate) before attempting REMOTE LOGIN. ckcmai.c, 10 Mar 2004. + +Added ckuker.nr to x.sh (ckdaily upload) and updated ckuker.nr with current +version number and dates. 10 Mar 2004. + +Replaced hardwired references to /usr/local in makefile with $(prefix) +(which defaults to /usr/local, but can be overridden on the command line), +suggested by Nelson Beebe for use with Configure. 10 Mar 2004. + +From Nelson Beebe: In the Kermit makefile in the install target commands, +line 981 reads: + + cp $(BINARY) $(DESTDIR)$(BINDIR)/kermit || exit 1;\ + +Could you please add this line before it: + + rm -f $(DESTDIR)$(BINDIR)/kermit;\ + +Some sites (mine included) keep multiple versions of software around, +with hard links between $(prefix)/progname and $(prefix)/progname-x.y.z. +Failure to remove the $(prefix)/progname at "make install" time then +replaces the old $(prefix)/progname-x.y.z with the new one, destroying +an old version that the site wanted to be preserved. makefile, 10 Mar 2004. + +Minor syntax and typo fixes (mostly prototypes): ckcdeb.h, ckcfns.c, +ckclib.c, ckufio.c, ckuusr.h, ckuusx.c, 10 Mar 2004. (I still have a few +more to do.) + +Added CC=$(CC) CC2=$(CC2) to many (but not all) makefile targets that +reference other makefile targets. On some platforms (notably AIX, Solaris, +SunOS) there are specific targets for different compilers, so I skipped +those. makefile, 10 Mar 2004. + +Added error checking to kermit:// URL macros, so they don't plow ahead +after the connection is closed. ckcmai.c, 11 Mar 2004. + +Added FreeBSD 4.9 and 5.1 targets (only the herald is affected). +makefile, ckuver.h, 11 Mar 2004. + +Added "LIBS=-lcrypt" to bsd44 targets since nowadays crypt is almost always +unbundled from libc. Also added explanatory notes. makefile, 11 Mar 2004. + +Changed MANDIR to default to $(manroot)/man/man1, and manroot to default +to $(prefix). More adding of CC=$(CC) clauses: {Free,Net,Open}BSD, 4.4BSD. +makefile, 11 Mar 2004. + +Miscellaneous cleanups: ckuusx.c, ckcnet.c, ckufio.c, 11 Mar 2004. + +Corrected the check in the linux target to see if /usr/include/crypt.h +exists, and if so to define HAVE_CRYPT_H, which is used in ckcdeb.h to +#include to get the prototype for crypt() and prevent bogus +conversions on its return type on 64-bit platforms (the previous test wasn't +quite right and the resulting symbol wasn't spelled right). makefile, +12 Mar 2004. + +From Jeff, 14 Mar 2004: + . Initialize localuidbuf[] in tn_snenv(): ckctel.c. + . Remove remote-mode checks in hupok() for K95G only (why?): ckuus3.c. + . Add help text for new K95-only TYPE /GUI switches: ckuus2.c. + . TYPE /GUI parsing, ...: ckuusr.c. + . TYPE /GUI action, dotype(): ckuus6.c + . Change Jeff's affiliation: most modules. + +20 Mar 2004: Looked into adding long file support, i.e. handling files more +than 2GB (or 4GB) long. Discovered very quickly this would be a major +project. Each platform has a different API, or environment, or transition +plan, or whatever -- a nightmare to handle in portable code. At the very +least we'll need to convert a lot of Kermit variables from long or unsigned +long to some new Kermit type, which in turn is #defined or typedef'd +appropriately for each platform (to off_t or size_t or whatever). Then we +have to worry about the details of open() vs fopen(); printf() formats (%lld +vs %Ld vs %"PRId64"...), platforms like HP-UX where you might have to use +different APIs for different file systems on the same computer, etc. We'll +need to confront this soon, but let's get a good stable 8.0.211 release out +first! Meanwhile, for future reference, here are a few articles: + +General: http://freshmeat.net/articles/view/709/ +Linux: http://www.ece.utexas.edu/~luo/linux_lfs.html +HP-UX: http://devrsrc1.external.hp.com/STK/partner/lg_files.pdf +Solaris: http://wwws.sun.com/software/whitepapers/wp-largefiles/largefiles.pdf + +Looked into FTP timeouts. It appears I can just call empty() (which is +nothing more than a front end for select()) with the desired timeout before +any kind of network read. If it returns <= 0, we have a timeout. This is +not quite the same as using alarm() / signal() around a recv() (which could +get stuck) but alarm() / signal() are not not used in the FTP module and are +not naturally portable to Windows, but select() is already in use in the FTP +module for both Unix and Windows. This form of timeout could be used +portably for both command response and data reads. What about writes to the +command or data socket? They can get stuck for hours and hours without +returning too, but the select() approach won't help here -- we need the +actual send() or recv() to time out, or be wrapped in an alarm()/signal() +kind of mechanism. But if we can do that for sends, we can also do it for +receives. Better check with Jeff before I start programming anything. +20 Mar 2004. + +Later: Decided to postpone the above two projects (ditto IPv6) until after +8.0.211 is released because both will have major impacts on portability. +Grumble: all i/o APIs should have been designed from the beginning with a +timeout parameter. To this day, hardly any have this feature. + +3-4 Apr 2004: More 8.0.211 Beta.02+ test builds: + + . FreeBSD 3.3 + . FreeBSD 4.4 + . Linux Debian 2.1 + . Linux RH 6.1 + . Linux RH 7.1 + . Linux RH 7.2 + . Linux RH 9 (with 84 different combinations of feature selection) + . Linux SuSE 6.4 + . Linux SuSE 7.0 + . NetBSD 1.4.1 + . NetBSD 1.5.2 + . OpenBSD 2.5 + . OpenBSD 3.0 + . QNX 4.25 + . SCO UnixWare 2.1.3 + . SCO UnixWare 7.1.4 + . SCO OpenServer 5.0.7 + . SCO XENIX 2.3.4 (no TCP) + +Changes needed: None. + +Problem: SCO XENIX 2.3.4 network build failed in the FTP module with +header-file syntax and conflicting-definitions trouble. I'm not going to +try to fix it; 8.0.209 built OK with FTP, so we'll just keep that one +available. + +Got access to VMS 8.1 on IA64. Building the nonet version of C-Kermit +required minor modifications to ckvvms.h, ckv[ft]io.c, and ckvcon.c, to +account for a third architecture. Also to SHOW FEATURES in ckuus5.c. Once +that was done, the UCX 5.5 version built OK too. Starts OK, makes Telnet +connection OK, sends files. Has some obvious glitches though -- "stat" +after a file transfer reports 0 elapsed time (in fact it was 00:09:48) and +1219174400 cps (when in fact it was 10364). This doesn't happen on the +Alpha. Btw, the IA64 binary is twice as big as the Alpha one. Changed +to Beta.03. 5 Apr 2004. + +Fixed the ckdaily script to include the makefile and man page in the Zip +file (they were not included because the Zip file was intended mainly for +VMS users, but some Unix users prefer Zip to tar.gz). 6 Apr 2004. + +Traced problems in VMS/IA64 statistics report to rftimer()/gftimer() in +ckvtio.c, which use sys$ and lib$ calls to figure elapsed time. These work +on VAX and Alpha but not IA64. Sent a report to the chief engineer of the +IA64 VMS port; he says it's probably a bug in VMS 8.1 (which is not a real +release); he'll make sure it's fixed in 8.2. As an experiment, tried +swapping in the Unix versions of these routines (which call gettimeofday() +etc). They seem work just fine (it hung a couple times but I think that's +because the underlying system hung too; trying it later on a new connection, +it was fine; however I noticed a BIG discrepancy in throughput between +sending and receiving). Moved definitions for VMS64BIT and VMSI64 to +ckcdeb.h so all modules can use them and added them to the SHOW FEATURES +display. Added VMSV80 definition to build procedure. Beta.03+. ckcdeb.h, +ckcuus5.c, ckcvvms.h, ckvtio.c, ckvker.com, 6 Apr 2004. + +While doing the build-all, I noticed the VMS version did not build with +Multinet or older UCX versions, always with the same errors -- undeclared +variables, undefined symbols, all TCP/IP related. This didn't happen a +couple weeks ago... Somehow the order of #includes was messed up -- +ckuusr.h depended on symbols that are defined in ckcnet.h, but ckcnet.h +was being included after ckuusr.h... this was compounded by two missing +commas in ckvker.com. 11 Apr 2004. + +Removed Beta designation, released as 8.0.211, 10 Apr 2004. + +I had somehow lost the edit to ckutio.c that changed the UUCP lockfile for +Mac OS X from /var/spool/uucp to /var/spool/lock. So I slipped it in and +re-uploaded version 8.0.211. You can tell the difference because SHOW +VERSIONS has 17 Apr 2004 for the Communications I/O module. Also the 10.3 +executable now has a designer banner: "Mac OS X 10.3". makefile, ckuver.h, +ckutio.c, ckuus[45].c, 17 Apr 2004. + +---8.0.211--- + +Removed "wermit" from "make clean" (how did it get there?). makefile. + +From Jeff, applied 10 May 2004. + . Rearrange #ifdefs that define OS/2-only features. ckcdeb.h. + . Fix two strncat()s that should have been ckstrncat()s. ckuus7.c. + . Fix two strncat()s that should have been ckstrncat()s. ckuus4.c. + . Fix one strncat(). ckcfns.c. + . SET FTP CHAR ON used backwards byte order when output to screen. ckcfns.c. + . Fix two strncat()s. ckuus3.c. + . Add SET NETWORK TYPE NAMED-PIPE for K95. ckuus3.c. + . Add "No active connections" message to hupok(). ckuus3.c. + . Fix many strncat()s. ckcnet.c. + . Fix some strncat()s. ckcftp.c + . Make FTP port unsigned short for 16383 < port < 65536. ckcftp.c. + . Improvements to FTP USER command. ckcftp.c. + . Fix FEAT parsing to allow for various forms of whitespace. ckcftp.c. + +S-Expression (AND FOO BAR) would not short-circuit if FOO's value was 0, +even though short-circuiting code has been there since Day 1. Similarly for +(OR BAR FOO). Turns out the first operand was a special case that bypassed +the short-circuit check. Fixed in dosexp(): ckuus3.c, 10 May 2004. + +Red Hat 7.3 (and maybe others) referenced open() without first +ensuring it was declared. The declaration is in , which is after + in ckutio.c series of #includes. Made a special case for this. +ckutio.c (see comments), 10 May 2004. + +If the local Kermit's parity is set to SPACE and then a file arrives via +autodownload, automatic parity detection improperly switches it to NONE. +Fixed in rpack() by switching parity automatically only if parchk() returns +> 0 (rather than > -1), since NONE and SPACE are indistinguishable. A +bigger problem still remains: autodownload does not work at all if the +sender is using actual parity bits (even, odd, or mark) and the receiver's +parity is NONE. ckcfn2.c, 10 May 2004. + +When a DIAL MACRO is defined and the phone number is comprised of more than +one "word" (i.e. contains spaces), the dial macro loses the second and +subsequent words after the first call. Fixed in xdial() by inserting quotes +around phone number before passing it to xdial(). ckuus6.c, 10 May 2004. + +DIAL MACRO fix was not right; the quotes were kept as part of the phone +number and sent to the modem. dodo() pokes its argument to separate the +macro argument string into its component arguments. xdial() is called +repeatedly on the same string, so after the first time, a NUL has been +deposited after the first word of the telephone number. The fix is to have +xdial() create a pokeable copy of its argument string before calling +dodo(dial-macro,args...). It might seem odd that dodo pokes its argument, +but making copies would be would be prohibitive in space and time. +ckuus6.c, 23 May 2004. + +FTP CD did not strip braces or quotes from around its argument. Fixed in +doftprmt(): ckcftp.c, 23 May 2004. + +Added client side of REMOTE MESSAGE/RMESSAGE/RMSG: ckuus[r27].c, 23 May 2004. + +Server side of REMOTE MESSAGE: ckcpro.w, 23 May 2004. + +From Dave Sneddon: an updated CKVKER.COM containing a fix where the +COMPAQ_SSL symbol was not defined but later referenced which generated an +undefined symbol error. ckvker.com, 5 Jan 2005. + +From Andy Tanenbaum (28 May 2005): + . Fix an errant prototype in ckcker.h and ckucmd.h - () instead of (void). + . Add support for MINIX 3.0. makefile, ckutio.c, ckufio.c, ckuver.h. + +Fixed messed-up sndhlp() call which apparently had been jiggered to +compensate for the bad prototype which has now been fixed, ckcpro.w, +12 Jun 2005. + +From Jeff (12 June 2005): + . Security updates. ck_ssl.c, ck_crp.c, ckuath.c. + . Fix bug in K95 SET PRINTER CHARACTER-SET. ckuus3.c. + . Add printer character-set to K95 SHOW PRINTER display. ckuus5,c + . Add SET MSKERMIT FILE-RENAMING to K95. ckuus7.c, ckuusr.h. + . Add help for K95 SET MSKERMIT. ckuus2.c. + . Add SET GUI CLOSE to K95. ckuusr.h, ckuus2.c, ckuus3.c + . Add help text for K95 SET GUI MENUBAR and TOOLBAR. ckuus2.c. + . Add --noclose command-line option for K95. ckuusy.c + . Add PAM support for Mac OS X. ckufio.c. + . Add GSSAPI support for Mac OS X. ckcftp.c. + . Pick up more URL options. ckcker.h, ckuusy.c. + . Fix bug in delta-time calculation across year boundary. ckucmd.c. + . Add Secure Endpoints to copyright notices. ckcmai.c. + . Fix FTP HELP to override unverbose setting. ckcftp.c. + . Fix assorted minor typos. + +From Matthias Kurz: automatic herald generation for NetBSD 2.0 and later, +"make netbsd2". ckuver.h, makefile, 12 Jun 2005. + +Added SET TERMINAL LF-DISPLAY, like CR-DISPLAY but for linefeed rather than +carriage return. ckuusr.h, ckuus[257x].c, 12 Jun 2005. + +Made a command-line option --unbuffered to do what the -DNONOSETBUF +compile-time option does, i.e. force unbuffered console i/o. Unix only. +ckuusr.h, ckuusy.c, ckutio.c, 12 Jun 2005. + +Fixed getiact() (which displays TERM IDLE-ACTION setting) to display +space as \{32}. ckuus7.c, 12 Jun 2005. + +Added LMV as a synonym for LRENAME, which is itself a synonym for LOCAL +RENAME. ckuusr.c, 12 Jun 2005. + +Put HELP SET TERMINAL DG-UNIX-MODE text where it belonged. ckuus2.c, +12 Jun 2005. + +Added IF LINK (Unix only) to test if a filename is a symlink. Uses the most +simpleminded possible method, calls readlink() to see if it succeeds or fails. +No other method is dependable across different Unixes. This code should be +portable because I already use readlink() elsewhere within exactly the same +#ifdefs. ckufio.c, ckuus2.c, ckuus6.c, 12 Jun 2005. + +Fixed a bug in which \fdir() wouldn't work when its argument was the nonwild +name of a directory file. zxpand(): ckufio.c, 12 Jun 2005. + +Made \fdirectory() a synonym for \fdirectories(). Made \fdir() an +acceptable abbreviation for these, even though it clashes with \fdirname(), +which still works as before. ckuus4.c, 12 Jun 2005. + +Added the long-needed \flopx() function, to return rightmost pieces of +strings, such as file extensions. \fstripx() and \flopx() are the +orthogonal functions we need to pick filenames apart from the right: +\stripx(foo.tar.gz) = foo.tar; flopx(foo.tar.gz) = gz. ckuusr.h, ckuusr.c, +ckuus2.c, 12 Jun 2005. + +Removed reference to defunct fax number, ckcmai.c, 12 Jun 2005. + +Added -DHAVE_PTMX to linux+krb5+openssl+zlib+shadow+pam. From Timothy Folks. +makefile, 12 Jun 2005. + +Built on Solaris 9 and NetBSD 2.0. + +From Jeff: New build target for Mac OS X 10.3 with Kerberos 5 and SSL. +makefile, 14 Jun 2005. + +Fixed error in ckuver.h NetBSD #ifdefs. 15 Jun 2005. + +Fixed SET TERMINAL IDLE-ACTION OUTPUT to work as documented, namely if the +output string is empty, to send a NUL. Previously there was no way to make +it send a NUL. ckuus7.c, 15 Jun 2005. + +Suppose (in Unix, for example) a filename contains wildcard characters, such +as {abc}.txt. When referring to such a file (e.g. in a SEND command), these +characters can be quoted, e.g. \{abc\}.txt. But if the file list has been +obtained programmatically, e.g. stored in an array, there is no way, short +of tedious, complicated, and error-prone string processing, to reference the +file. For this we need a way to disable wildcard processing. I added { ON, +OFF } choices for the SET WILD and SHOW FILE commands: ckuusr.h, ckuus[234].c. +{ ON, OFF } turns wildcarding off and on without affecting the { KERMIT, +SHELL } agent choice; it does this by setting a new and separate global +variable, wildena. Added semantics to ckufio.c. Crude but effective. It +might have been more Unixlike to add Yet Another form of quoting but we +have enough of that already (later maybe I'll add a \function() for this). +Needs to be propagated to Windows and VMS. 15 Jun 2005. + +Improved and fixed typos in HELP WILDCARD and HELP PATTERN. ckuus2.c, +15 Jun 2005. + +The GREP command, and probably anything else that uses ckmatch() for pattern +matching, failed on patterns like */[0-3]*.html. The [a-b] handler, when +failing to match at the current position, neglected to back up the pattern +and try again on the remainder of the string. I also fixed another case, in +which matching a literal string a*b?c against the pattern a[*?]*[?*]c caused +ckmatch() to recurse until it blew up. ckclib.c, 16 Jun 2005. + +Added builds and designer banner for Solaris 10. makefile, ckuver.h, +27 Jun 2005. + +Defined CKHTTP for NetBSD, the HTTP code builds and works fine there. +ckcdeb.h, 2 Jul 2005. + +Added #ifndef OSF40..#endif around definition of inet_aton() in ck_ssl() +to allow building in Tru64. Added tru64-51b+openssl to makefile. +15 Jul 2005. + +HTTP GET would fail if the URL contained any metacharacters, no matter how +much you quoted them. Although it uses cmfld() to parse the (partial) URL, +it then uses cmofi() to get the output filename, which by default is the +"filename" from the URL, which might be something like "rankem.asp?id=1639". +cmofi() refuses to accept unquoted metacharacters in "filenames" and that's +what happens in this case if the output filename is not specified. Worked +around this by disabling wildcard processing around HTTP GET using the new +"wildena" variable from June 15th. ckuusr.c, 18 Jul 2005. + +Fixed the June 16th fix to the pattern matcher. I fixed a real problem, but +I made an unrelated optimization that introduced new ones. ckclib.c, +18 Jul 2005. + +Added missing help text for \fb64encode() and \fb64decode(). ckuus2.c, +18 Jul 2005. + +Changed SET WILD OFF help text to warn that this setting prevents the +creation of backup files (later I'll have to see if something more useful +can be done about this). ckuus2.c, 18 Jul 2005. + +Built OK on Mac OS X 10.4.2 using macosx103 target (but with some +"signedness" warnings in ckcnet.c and ckcftp.c). Built on Unixware 7.1.4 +with uw7 target. 27-28 Jul 2005. + +Added -DCKHTTP to Mac OS X 10.3-.4 KFLAGS. Makefile, 4 Aug 2005. + +Built on BSDI 4.3.1. Added -DCKHTTP. + +Compact substring notation extended to accept not only start:length but also +start-end notation. Thus \s(foo[12:18]) means the substring of foo starting +at position 12 of length 18, and tne new \s(foo[12-18]) means the substring +of foo starting at position 12 and ending with position 18. Ditto for +\:(\%a), etc. ckuus4.c, 9 Aug 2005. + +See correspondence with Mark Sapiro, Nov 2003 and Sep 2004, about certain +variations on IF syntax having been broken by the introduction of "immediate +macros" circa 1999. It seems the problem -- variables not being expanded -- +always occurs in the ELSE part when (a) the IF condition is false; (b) the +ELSE command is "standalone", i.e. expressed as a separate command after the +IF command (original C-Kermit 5A syntax), and (c) its command list is a block. +This would suggest the problem is in the XXELS parser. + +Going back to 1999, I find this: + Fixed a problem Jim Whitby noticed with quoting in ELSE statements. This + problem was introduced when I unified IF and XIF, and occurs only when + ELSE begins on a line, followed by a { command list } rather than a single + command. The solution (gross) was to make a special version of pushcmd() + (called pushqcmd()) for this situation, which doubles backslashes while + copying, BUT ONLY IF it's a command list (i.e. starts with "{"); otherwise + we break lots of other stuff. Result passes Jim's test and still passes + ckedemo.ksc and iftest.ksc. ckucmd.c, ckuus6.c, 27 Sep 99. + +I undid this change and it made no difference to all the other IF +constructions (in fact, it fixed an unrelated one that was broken, so now +iftest scores 54 out of 54, instead of 53). However, it does not fix the +ELSE problem; in fact it pushes it all the way in the other direction: + + The opposite occurs any time you try to execute an immediate macro inside a + macro or any other { block }: not only is the variable evaluated, it is + evaluated into nothing. It looks like this happens only in immediate + macros, i.e. *commands* that start with '{'. So maybe we really have two + isolated problems, that can each be fixed. + +The situation is illustrated by this simple script: + + def xx { + if false { echo \%1, echo \%2 } + else { echo \%3, echo \%4 } + } + xx one two three four + +With pushqcmd() it echoes the variable names literally; with pushcmd() it +echoes empty lines. Since ELSE, when its argument is a block, dispatches +to the immediate-macro handler, it seems we have unified the two problems, +so fixing one should fix the other. + +The problem is that we define a new temporary macro and then call dodo() to +execute it. But if the definition contains macro arguments, we have added a +new level of macro invocation, thus wiping out the current level of args. +The cure is to expand the variables in the immediate macro in the current +context, before executing it. This means simply changing the cmtxt() call +that reads the immediate macro to specify xxsting as its processing +function, rather than NULL, which is used for real macros to defer their +argument evaluation until after the macro entered. ckuusr.c, 11 Aug 2005. + +Added a new makefile target, macosx10.4, for Mac OS X 10.4. This one uses +an undocumented trick to get the otherwise unavailable-except-by-clicking +Mac OS X version number (in this case 10.4.2) and stuff it into the HERALD +string. makefile, 11 Aug 2005. + +Built OK on Solaris 9, Solaris 10 (with a few implicit declaration warnings +in ckuusx.c), Mac OS X 10.4.2 (with some warnings in ckcnet.c and ckcftp.c), +Mac OS X 10.3.9 (also using the macos10.4 entry, which gets the right +version number, and gets no warnings at all), RH Enterprise Linux AS4 on AMD +x86_64, Tru64 Unix 4.0F, SCO UnixWare 7.1.4 + +For docs and/or scriptlib: Unix C-Kermit can be a stdin/out filter. The +trick is to use the ASK, ASKQ, or GETC command for input, specifying no +prompt, and ECHO or XECHO for output, e.g.: + +while true { + ask line + if fail exit 0 + echo \freverse(\m(line)) +} +exit 0 + +FOPEN didn't do anything with the channel number if the open failed, so any +subsequent command that tried to reference it would get a parse error it was +undefined or non-numeric, not very helpful. Changed FOPEN to set the +channel number to -1 if the file can't be opened. Now subsequent operations +on the channel fail with "Channel -1: File not open". I also added two +magic channel numbers: -8 means that any FILE command (besides OPEN and +STATUS) on that channel is a noop that succeeds silently; -9 is a noop that +fails silently. So now it's possible to simply set a channel number to one +of these values to disable i/o to certain file without getting lots of error +messages. dofile(): ckuus7.c, 12 Aug 2005. + +Added automatic herald construction for UnixWare 7. makefile, 12 Aug 2005. + +Unix isdir() never allowed for arguments that started with tilde, so gave +incorrect results for ~/tmp/ or ~fdc. The problem was mainly invisible +since most commands that parsed file or directory names used cmifi(), cmdir(), +etc, which did the conversions themselves. But IF DIRECTORY was an exception, +since its operand had to be treated as just text, and then tested after it +was parsed. ckufio.c, 13 Aug 2005. + +Fixed the following: +"ckuusx.c", line 8959: warning: implicit function declaration: ckgetpeer +"ckufio.c", line 1869: warning: implicit function declaration: ttwait +"ckufio.c", line 2941: warning: implicit function declaration: mlook +"ckufio.c", line 2943: warning: implicit function declaration: dodo +"ckufio.c", line 2944: warning: implicit function declaration: parser +"ckcftp.c", line 2625: warning: implicit function declaration: delta2sec +"ckcftp.c", line 4071: warning: no explicit type given for parameter: prm +"ckcftp.c", line 8389: warning: no explicit type given for parameter: brief +ckuusx.c, ckufio.c, ckcftp.c, ckucmd.h. 13 Aug 2005. + +Unbuffered stdout code has never worked because the setbuf(stdout,NULL) call +has to occur before the stdout has been used. The reason it's needed is +that some Kermit code writes to stderr (which is unbuffered) and other code +writes to stdout, and therefore typescripts can come out jumbled. Robert +Simmons provided the needed clue when he insisted it +worked only when executed at the very beginning of main(). So I moved the +code to that spot. But since now we also want to make unbuffered a runtime +(command-line) option, I had to do a clunky by-hand pre-prescan inline in +main() to look thru argv[], even before prescan() was called. ckcmai.c, +ckutio.c, ckuusy.c, 13 Aug 2005. (Now that this works, it might be a good +idea to remove all use of stderr from Kermit.) + +Managed, after some finagling, to build a 64-bit version on Solaris 10 at +Utah Math with Sun cc. (Can't make any gcc builds at all, 32- or 64-bit, +they all blow up in .) New target: solaris10_64. makefile, +15 Aug 2005. + +The 64-bit Solaris 10 version compiles and links OK and transfers files in +remote mode. It can make FTP connections and use them, but Telnet connections +always fail with "network unreachable". This is with all default libs and +include files. Nelson has a separate set in /usr/local, which he references +explicitly in all his 64-bit builds, but using these makes no difference. +Some data type is wrong in ckcnet.c. But telnet works fine in 64-bit Linux +and Tru64 builds. Debug logs trace the difference to netopen() (of course), +the spot where we test the results of inet_addr(), which is already marked +suspicious for 64-bit builds. It seems that inet_addr() is of type in_addr_t, +which in turn is u_int32, i.e. an unsigned 32-bit int. Yet the man page says +that failure is indicated by returning -1. I guess this doesn't matter in +32-bit builds, but in the 64-bit world, the test for failure didn't work +right. I made a Solaris-specific workaround, and checked that it works in +both 32-bit and 64-builds. I really hate typedefs. ckcnet.c, 15 Aug 2005. + +Changed the plain-text version (as opposed to the popup or GUI version - the +GUI version, at least, already does this) of ASKQ to echo keystrokes +asterisks rather than simply not echo anything, so it's easier to see what +you're doing, the effects of editing, etc. Experimental; for now, there's +no way to disable this. Not sure if there needs to be. Anyway, to get this +working required a fair amount of cleaning up of gtword(), which was echoing +different ways in different places. ckuus6.c, ckucmd.c, 15 Aug 2005. + +Added a solaris9_64 target for building a 64-bit version on Solaris 9 with +Sun cc. Verified, using the DIR command and \fsize() function on a 4.4GB +file, that the Solaris 64-bit version of Kermit gets the size correctly, and +that it can copy such a file (thus its fopen/fread/fwrite/fclose interface +works right). Initiated a large-file transfer between here and Utah over +SSH and verified that it puts the correct file size in the A packet when +sending; the right quantities are shown on the file transfer display (file +size CPS, percent done, etc). But even at 5Mb/sec, it takes a good while to +transfer 4.4GB, more than 2 hours (not streaming; 30 window slots, 4K +packets, maybe it would go faster with streaming)... After an hour or so, +it filled up the partition and gave up (gracefully) before it reached the +2GB frontier (drained its pending packets, closed the partial file). +Restarted at 12:54, this time with streaming and 8K packets (the speed +wasn't significantly different). This time it transferred 95% of the file +(4187660288 bytes) before failing because the disk filled up. Went to Utah +and started a transfer between two Solaris 10/Sparc hosts; this goes about 8 +times faster. The transfer completed successfully after 17m41s. All fields +in the f.t. display looked right the whole time. Then I verified various +other 64-bit combinations transferring the same 4.4GB file: + + To................ + From Sol Amd i64 Tru + Sol OK OK OK OK Sol = Solaris 10 / Sparc + Amd OK Amd = AMD x86_64 RH Enterprise Linux AS4 + i64 OK i64 = Intel IA64, RH 2.1AS + Tru Tru = Tru64 Unix 4.0F Alpha + +(The other combinations are difficult to test for logistical reasons.) + +Tried sending the same long file with Kermit's FTP client. It chugged along +for a while until I stopped it; it would have taken hours to complete. +There is no indication that it wouldn't have worked, assuming the FTP server +could also handle long files, which who knows. Anyway, Kermit showed all +the right data on the display screen. 17 Aug 2005. + +On AMD x86_64 and IA64 native 64-bit Linux builds, the pty routines did not +work at all. ptsname() dumped core. If I commented out ptsname(), then the +next thing dumped core. The same code works on the other 64-bit builds. +Poking around, I see that this version of Linux has an openpty() function, +which I could try using instead of the current API -- grantpty(), etc. Then +I see that openpty() is already coded into Kermit's pty module, +conditionalized under HAVE_OPENPTY, which has never before been defined for +any build. I added a test to the makefile linux target (look for the +openpty() prototype in , if found define HAVE_OPENPTY as a CFLAG and +also add -lutil to LNKFLAGS). Works fine on the problem builds, and also +on previously working 32-bit builds. makefile, 17 Aug 2005. + +Fixed a bug in the ASKQ echo asterisks code, which made the VMS version of +C-Kermit always echo asterisks. Turns out that some code in the main parse +loop to reset command-specific flags was in the wrong place, which had other +effects too, for example ASKQ temporarily turns off debug logging as a +security measure, but the code to turn it back on was skipped in most cases. +Some other side effects related to the DIRECTORY and CD commands might have +been possible but I haven't seen them. ckuus[56].c, 23 Aug 2005. + +Problem reported when sending a file to VMS when the name in the F packet +starts with a device specification and does not include a directory field, +and PATHNAMES are RELATIVE. Example: dsk:foo.bar becomes f_oo.bar. The +code assumes that if there is a device field, it is followed by a directory +field, and it inserts a dot after the '[', which in this case is not there. +Later the dot becomes '_' because of the only-one-dot rule. Solution: only +insert the dot if there really is an opening bracket. nzrtol(): ckvfio.c, +23 Aug 2005. + +A report on the newsgroup complains that C-Kermit and K95 servers were +sending REMOTE DIR listings with only #J line terminators, rather than #M#J. +Yet all the other REMOTE xxx responses arrived with #M#J. snddir() was +neglecting to switch to text mode. ckcfns.c, 26 Aug 2005. + +Back to long files. What happens if 32-bit Kermit is sent a long file? +It gets an A-packet that looks like this: + + ^A_"A."U1""B8#120050815 18:28:03!'42920641*4395073536,#775-!7@ )CP + +The 32-bit receiver reacts like so: + + gattr length[4395073536]=100106240 + +the first number being the string from the A-packet, the second being the +value of the long int it was converted to by atol(). Clearly not equal in +this case. When this happens Kermit should reject the file instead of +accepting it and then getting a horrible error a long time later. Added +code to gattr() to convert the result of atol() back to a string and compare +it with the original string; if they're not equal, reject the file on the +assumption that the only reason this could happen is overflow. Also some +other code in case the sender sends the only LENGTHK attribute. Now files +whose lengths are too big for a long int are rejected right away, provided +the sender sends the length in an A packet ahead of the file itself. If +this new code should ever cause a problem, it can be bypassed with SET +ATTRIBUTE LENGTH OFF. ckcfn3.c, 26 Aug 2005. + +As I recall from when I was testing this a few weeks ago, when the too-big +length is not caught at A-packet time, the transfer fails more or less +gracefully when the first attempt is made to write past the limit. I went +to doublecheck this by sending a big file from the 64-bit Solaris10 version +to a 32-bit Mac OS X version that does not have today's code. The Mac +thinks the incoming file is 2GB long when it's really 4GB+. But in this +case, something new happens! Although the percent done and transfer rate go +negative, the file keeps coming. It would seem that Mac OS X lets us create +long files without using any special APIs. The transfer runs to completion. +Mac OS X Kermit says SUCCESS (but gets the byte count and cps wrong, of +course). But then a STATUS command says FAILURE. The file was, however, +transferred successfully; it is exactly the same length and compares byte +for byte with the original. This tells me that in the Mac OS X version -- +and how many others like it??? -- today's rejection code should not be +enabled. Meanwhile I put today's new code in #ifndef NOCHECKOVERFLOW..#endif, +and defined this symbol in the Mac OS X 10.4 target. Over time, I'll have +to find out what other platforms have this characteristic. And of course +I'll also have to do something about file-transfer display, statistics, and +status. makefile, ckcfn3.c, 26 Aug 2005. + +From now on I'm going to bump the Dev.xx number each time I upload a new +ckdaily. This one will be Dev.02. ckckmai.c, 26 Aug 2005. + +Got rid of all the extraneous FreeBSD 4 and 5 build targets. Now there's +one (freebsd) for all FreeBSD 4.1 and later. makefile, 27 Aug 2005. + +Mac OS X 10.4 (Tiger) is a 64-bit OS. Building C-Kermit 0n 10.4.2 without +any special switches stilll gives a 32-bit executable. Ditto building with +-mpowerpc64. Further investigation turned up a tip sheet on MySQL that says +you have to include all of these: -mpowerpc64 -mcpu=G5 -mtune=G5 -arch +ppc64. That did the trick. New makefile target: macosx10.4_64. But the +10.4.2 system I tried did not have 64-bit [n]curses or resolv libs, so this +build has no -DNOCURSES -DNO_DNS_SRV. makefile, 27 Aug 2005. + +Created a symbol CK_64BIT to indicate true 64-bit builds at compile time. +Added 64-bit announcement to the startup herald and the VERSION text. +ckcdeb.h, ckuus[r5].c, 27 Aug 2005. + +Added a built-in variable \v(bits) to indicate the size of the build +(16, 32, 64, or whatever else sizeof() might report). ckuusr.h, ckuus4.c, +27 Aug 2005. + +Got rid of all the warnings in 64-bit Mac OS X about args to getsockopt(), +getsockname(), and getpeername(), and the comparisons on the return value +of inet_addr(). ckcnet.[ch], 27 Aug 2005. + +Now to check the effects on other builds... + Linux on AMD64: ok. + Linux on IA64: ok. + Linux on i386: ok. + Mac OS X 10.3.9 32-bit: ok. + Solaris 10 64-bit: ok. + Solaris 9 32-bit: ok. + Tru64 4.0F: ok. + FreeBSD 4.11: ok. + FreeBSD 5.4 ia64 (64-bit): ok. + FreeBSD 5.4 i386 (32-bit): ok. + +The Tru64 5.1B build totally blew up because they have their own unique +sockopt/etc length-argument data type (int!), so I had to roll back on using +socklen_t for this in all 64-bit builds. Checked to make sure it still +builds on Tru64 4.0F after this change (it does). ckcnet.h, 27 Aug 2005. + +The HP-UX 11i/ia64 build comes out to be 32-bit but thinks it's 64-bit. +CK_64BIT is set because __ia64 is defined. So how do I actually make a +64-bit HP-UX build? I tried adding +DD64 to CFLAGS, and this generates +64-bit object files but linking fails to find the needed 64-bit libs +(e.g. -lm). For now I added an exception for HPUX to the CK_64BIT +definition section. ckcdeb.h, 27 Aug 2005. + +Took the time to verify my recollection about the "graceful failure" on a +regular Pentium Linux system when receiving a too-big file... OK, it's not +exactly graceful. It gets a "File size limit exceeded" error; the message +is printed in the middle of the file-transfer display, apparently not by +Kermit, and Kermit exits immediately. Looks like a trap... Yup. "File +size limit exceeded" is SIGXFSZ (25). What happens if we set it to SIG_IGN? +Just the right thing: The receiver gets "Error writing data" at 2147483647 +bytes, sends E-packet to sender with this message, and recovers with total +grace (drains packet buffers, returns to prompt). ckutio.c, 27 Aug 2005. + +Backed off from rejecting a file because its announced size overflows a +long. Now instead, I set the file size to -2 (a negative size means the +size is unknown, but we have always used -1 for this; -2 means "unknown and +probably too big"). In this case, the f-t display says: + + File Size: POSSIBLY EXCEEDS LOCAL FILE SIZE LIMIT + +then the user can interrupt it with X or whatever, or can let it run and +see if maybe (as in the case of Mac OS X) it will be accepted anyway. This +way, we skip all the bogus calculations of percent done, time remaining, etc. +ckcfn3.c, ckuusx.c, 27 Aug 2005. + +Discovered that VMS C-Kermit on Alpha and IA64 is a 32-bit application; +sizeof(long) == sizeof(char *) == 4. Tried adding /POINTER_SIZE=64 to VMS +DECC builds on Alpha and IA64, but the results aren't great. Tons of +warnings about pointer size mismatches between Kermit pointers and RMS ones, +and the executable doesn't run. It appears that access to long files +would require a lot of hacking, similar to what's needed for 32-bit Linux. + +--- Dev.02: 27 Aug 2005 --- + +From Jeff, 28 Aug 2005. + . Fix SSH GLOBAL-KNOWN-HOSTS-FILE / USER-KNOWN-HOSTS-FILE parsing, ckuus3.c. + . Pick up K95STARTFLAGS from environment, ckuus4.c. + . Fix some typos in command-line processing (-q), ckuus4.c. + . Be sure to suppress herald if started with -q, ckuus7.c. + . Fix ssh command-line switches, ckuusy.c. + +Eric Smutz complained that HTTP POST was adding an extraneous blank line, +which prevented his application from successfully posting. RFC 2616 states +(in Section 4.1): + + In the interest of robustness, servers SHOULD ignore any empty + line(s) received where a Request-Line is expected. In other words, if + the server is reading the protocol stream at the beginning of a + message and receives a CRLF first, it should ignore the CRLF. + + Certain buggy HTTP/1.0 client implementations generate extra CRLF's + after a POST request. To restate what is explicitly forbidden by the + BNF, an HTTP/1.1 client MUST NOT preface or follow a request with an + extra CRLF. + +This seems pretty clear. One section of code in http_post() (just above the +postopen: label) was appending a CRLF to a buffer whose last already was +terminated by CRLF, and then appended a second CRLF; thus two empty lines. +I removed the second one. ckcnet.c, 28 Aug 2005. + +I looked into the 64-bitness of NetBSD, it seems to be like Linux and +FreeBSD on 64-bit hardware, i.e. you just build it there and it works, at +least on Alpha and AMD64, going back to NetBSD 1.4 or 1.5. But I don't have +access to any of these for verification and documentation on the Web is +scanty. + +Checked PeterE's complaint again of warnings in ckutio.c about parameter +list of get[ug]id() and gete[ug]id(). When I "make hpux1100o" on HP-UX +11.11 (PA-RISC), there are definitely no warnings. He says the same thing +happens on 10.xx, but I don't have access to that any more. I also did +"make hpux1100o" on HP-UX 11.23 (11i v2) (PA-RISC), also no warnings. +(Except in both cases, a warning about a comment within a comment in +/usr/include/sys/ptyio.h). On HP-UX 11i v2 on Itanium, however, there are +TONS of warnings, mostly of the "variable set but never used" kind. Also +"dollar sign used in identifier". Tracking this last one down, I see it's +complaining about code that's in #ifdefs for other platforms, such as +Apollo Aegis. Is "aegis" defined in HP-UX 11i v2/IA64? No! (It would show +up in SHOW FEATURES if it was.) Some phase of the compiler is complaining +about code that it should be skipping (and that, in fact, it *is* skipping +it because the build is successful). It's as if cc is running lint for me +but not telling lint which macros are defined and which are not. + +Verified that 64-bit linking fails in the same way for HP-UX 11i v2 on both +IA64 and PA-RISC. Sent a query to HP. + +Compiling ckcnet.c and ckcftp.c got the familiar sockopt-related warnings on +HP-UX 11i v2; turns out it is just like Tru64 Unix in using an int for the +length argument. Added another special case and the warnings went away. +ckcnet.h, 28 Aug 2005. + +Added some stuff to SHOW FEATURES to see what kinds of macros are exposed +(e.g. INT_MAX, LONG_MAX, LLONG_MAX, etc) and also show sizeof(long long) and +sizeof(off_t). Building this code all over the place will give me an idea +of how widespread these data types are, and to what extent I can tell +whether they are available from clues in the header files. (At first +glance, it appears that I'm not picking up , but adding an +#include for it is just asking for trouble.) No complaints about long long +or off_t from Solaris 9 or recent Linuxes. ckuus5.c, 28 Aug 2005. + +Fixed a warning in HP-UX 10 and 11 stemming from some old-style prototypes +in ckutio.c for get[re][gu]id(). ckutio.c, 29 Aug 2005. + +Updated minix3 target from Andy Tanenbaum. makefile, 29 Aug 2005. + +PeterE confirms that "long long" and off_t are available in all HP-UX 10 and +11, and in HP-UX 9 on PA-RISC but not Motorola. 30 Aug 2005. + +Got 64-bit builds to work on HP-UX. According to my notes, John Bigg of HP +said (in 1999) that HP-UX 10.30 and later require PA-RISC 1.1, and do not +work on PA-RISC 1.0. But is PA 1.0 64-bit or what? Today, Alex McKale of +HP said "The 64-bit binaries will work on all machines that have the same or +later release of HP-UX (excluding PA-RISC 1.1 machines)". Still need +clarification... Maybe it's that all IA64 builds can be 64-bit but I need +dual builds for PA-RISC. Meanwhile I started transfer of a 4GB+ file from +Solaris to HP-UX 11i but it exceeded some quota on the HP long before it +approached the 2G point. It failed cleanly and up until then it was working +fine (numbers, stats, etc). 30 Aug 2005. + +Support of large files in 32-bit builds began in 10.20. 64-bit application +support began in 11.00, but not all machines that run 11.00 support 64 bits. +About long files, see HP /usr/share/doc/lg_files.txt. + +PeterE found that certain patterns can still make Kermit loop; example: + + if match T01011-00856-21-632-073 *[abc] { echo GOOD } else { echo BAD } + if match T01011-00856-21-632-073 *[a-z] { echo GOOD } else { echo BAD } + +The minimum offending pattern is * followed immediately by an [xxx] +construction, followed by anything else, including nothing. Previous +versions of Kermit handled this one correctly, without looping (but failed +certain matches that should have succeeded). The new section of code I +added on 15 June, upon failure to match, advances the string pointer and +backs up the pattern to the previous pattern, and starts again +(recursively). However, there needed to be a corresponding check at entry +for an empty target string. ckmatch(): ckclib.c, 12 Sep 2005. + +PeterE discovered that "kermit -y filethatdoesnotexit" gives an erroneous +error message that names the user's customization, rather than the name +given on the command line. doinit(): ckuus5.c, 12 Sep 2005. + +FREAD does not get an error if it tries to read a record or file or piece of +file that is too big for its buffer. In particular, FREAD /SIZE:xxx seems +to succeed even if less than xxx was read. It should fail unless, perhaps, +it successfully read up to the end of the file. Furthermore, if xxx is +bigger than the file buffer size, it should complain. The buffer is +line[LINBUFSIZ], 32K. The lack of failure was due to code in dofile() that +adjusted the given size silently if it was greater than the buffer size, +which I removed, and also added a check when parsing the /SIZE: switch. +dofile(): ckuus7.c, 12 Sep 2005. + +That still didn't help with FREAD /SIZE:n returning less than n bytes, even +when they were available. That's because the underlying routine, z_in(), +didn't check fread()'s return code, which is the number of bytes read. +If fread() has smaller buffers, it needs to be called in a loop. z_in(): +ckuus7.c, 12 Sep 2005. + +Flen() fails on strings of length 8192 or more. The limitation is in the +callers of zzstring, which seem to be specifying an 8K buffer, in this case +fneval(). The operable symbols are FNVALL (max length of value returned by +a function) and MAXARGLEN (maximum length of an argument to a function). I +changed both of these for BIGBUFOK builds to be CMDBL. Buffers can never be +infinite, there has to be a limit. It's important to make everything work +consistently within that limit, and to make something useful happen when the +limit is exceeded. At this point, I can probably also increase the limits +for modern 32-bit systems, and certainly for 64-bit ones. Also there's no +point in worrying about 16-bit platforms any more; earlier C-Kermit versions +can still be used on them if necessary. ckuusr.h, 12 Sep 2005. + +Special #ifdefs for finding resolv.h and nameser.h in MINIX3 from Andy +Tanenbaum. ckcnet.c, 20 Sep 2005. + +PeterE noticed that ckmatch(), even though it works pretty well now, does a +lot of extra and unnecessary recursion after determining the string and +pattern do not match, at least when the pattern is of the form *[abc]. +After several false starts I was able reduce this effect to a minor level +(but not eliminate it all together) by changing a while loop into a do loop. +ckmatch(): ckclib.c, 15 Oct 2005. + +Added -DNOLONGLONG to HP-UX 8.00 and earlier builds, and to Motorola-based +HP-UX 9.00 builds. This is simply to inhibit the test for whether "long +long" is supported by the compiler, since when it isn't, the module +containing the test won't compile. makefile, ckuus5.c, 16 Oct 2005. + +Making ASKQ always echo asterisks is a bad idea, because when it doesn't +echo, it's the perfect way to read silently from stdin, e.g. in a CGI script +(INPUT can also be used for this but it's not as straightforward). So I put +the default for ASKQ back to no echoing, then gave ASKQ its own switch +table, which is the same as for ASK with the addition of an /ECHO:x switch, +which tells what character to echo. ckucmd.c, ckuus[26].c, 17 Oct 2005. + +Fixed a bug in FTP GET /COMMAND filename commandname; it always dumped core +dereferencing a null string (the nonexistent local asname). ckcftp.c, +17 Oct 2005. + +For docs: if you don't like the funny business that happens when you type +an IF command at the prompt, use XIF instead and it won't happen. Also note +that commands like "if xxx { echo blah } else { echo blah blah }" don't +work when typed at the prompt; you have to use XIF for this. + +Back to ckmatch()... Under certain conditions (e.g. patterns like *[abc]) +failure to match would not stop the recursion because the string and pattern +arguments are on the stack, as they must be, so there was no way for level +n-1 to know that level n had detected a definitive nonmatch and that no +further attempts at matching were required. The right way to handle this is +to recode the whole thing as coroutines, the cheap way out is with a global +static flag. Works perfectly, in the sense that the match.ksc test results +are identical to what they were before and the extra backing up and +recursion are eliminated. (The Oct 15th fix wasn't really a fix, it broke +a couple of cases.) ckclib.c, 20 Oct 2005. + +ckuus7.c(2987): warning #267: the format string requires additional arguments +(in PURGE command); fixed 20 Oct 2005. + +From Andy Tanenbaum, final changes for MINIX3: #ifdef out the inline +definitions for gettimeofday() and readlink(). ckutio.c, 23 Oct 2005. + +From Jeff: struct gss_trials initializers changed from gss_mech_krb5 to +ck_gss_mech_krb5. ckcftp.c, 23 Oct 2005. + +From Jeff: some improvements to K95 GUI SHOW TERMINAL. ckuus5.c, 23 Oct 2005. + +Found and corrected some misplaced #ifdefs in shofeat(), ckuus5.c, 23 Oct 2005. + +--- Dev.03 --- + +Fixed a compiler warning in a debug() statement in zzstring() by adding +parens. ckuus4.c, 24 Oct 2005. + +Added -DNOLONGLONG to sv68r3v6 target, makefile, 25 Oct 2005. + +New makefile targets for HP-UX from PeterE to handle the 'long long' +situation. 26 Oct 2005. + +From Jeff: changes to support OpenSSL 0.9.8, ck_ssl.h. ckcasc.h has had +short names defined for ASCII control characters for 20-some years but now +they are causing conflicts, so EM becomes XEM (also for OpenSSL 0.9.8). +Changed K95's default terminal type from VT320 to VT220 because VT320 +termcaps/terminfos are disappearing from Unix hosts: ckuus7.c. Reorganize +the data-types section of SHOW FEATURES to add more macro tests for integral +sizes and to provide for the proper printf formatting in order to allow the +sizes to be output ("You are going to need to be careful because %llx is not +supported on all platforms. On Windows, it is the same as %lx, 32 bits"): +ckuus5.c, 26 Oct 2005. + +Defined NOLONGLONG ckcdeb.h for various old platforms where we know we are +never going to need 64-bit ints (even if they support a long long datatype, +chances are pretty slim they supported 64-bit file sizes). ckcdeb.h, +26 Oct 2005. + +PeterE noticed that GOTO targets can only be 50 characters long. This was +by design, a long time ago, on the assumption that nobody would make longer +labels. But in SWITCH statements, case labels can be variables that expand +to anything at all. If we chop them off at 50, we might execute the wrong +case. Changed the maximum label size to be 8K, and added code to dogoto() +to check when a label or target is too long and fail, to prevent spurious +GOTO or SWITCH results. ckuusr.h, ckuus[r6].c, 26 Oct 2005. + +Testing revealed there was still a problem with SWITCH case labels that were +variables that expanded into long strings. Turns out that I was being +too clever when I decided that, if the SWITCH macro was n1 characters long +and the case-label search target was n2 characters long, I only had to +search the first n1-n2+1 characters of the macro definition. That was true +before I allowed case labels to be variables, but not any more! Fixed in +dogoto(): ckuus5.c, 26 Oct 2005. + +--- Dev.04 --- + +Dev.04 didn't actually contain Jeff's data-type changes to shofeat(), +I think I saved the wrong buffer in EMACS... Fixed now. 27 Oct 2005. + +PeterE corrected a typo in the HP-UX 7.00 makefile target. 27 Oct 2005. + +PeterE had been reporting problems stress-testing the new SWITCH code, but +only on HP-UX 9, primarily stack overrun. Turns out to be the HP-UX 9 +optimizing compiler's fault. No optimization, no problems. + +PeterE found that even when dogoto() detects a string that is too long +and fails, this does not stop SWITCH from producing a result, which can not +possibly be trusted. Changed the part of dogoto() that handles this to +not just fail, but also to exit the script immediately and return to top +level. ckuus6.c, 28 Oct 2005. + +An idea popped into my head after having typed too many commands like "dir +ck[cuw]*.[cwh]" to check the list of matching files, and then having to +retype the same filespec in a SEND command: Why not unleash some unused +control character such as Ctrl-K to spit out the most recently entered input +filespec? It was easy, just a few lines in cmifi2() and gtword(), plus a +couple declarations. To see all the changes, search for "lastfile" (all the +new code is protected by #ifndef NOLASTFILE). ckucmd.c, 28 Oct 2005. + +I added a new variable \v(lastfilespec) that expands to the same last +filespec, for use in scripts. ckuusr.h, ckuus4.c, 28 Oct 2005. + +The Unix version of C-Kermit failed to put anything in the session log if +SET TERMINAL DEBUG ON. Rearranged the pertinent clause so logging happens +independent of TERMINAL DEBUG. For now, since the user who noticed this +wanted debug format to go into the session log, that's what I do. The +alternative would be to just log the raw incoming stream as usual, or to add +Yet Another SET Command to choose. ckucns.c, 11 Nov 2005. + +Fixed HELP INTRO text. ckuus2.c, 11 Nov 2005. + +Added NOLONGLONG for SV68. ckcdeb.h, 11 Nov 2005. + +--- Dev.05 --- + +Added a debug() statement in FTP secure_getbyte() to see what's going on +with Muhamad Taufiq Tajuddin's 205-byte-per-second FTP/SSL downloads. + +--- Dev.06 --- + +Result: nothing, SSL_get_error() does not report any errors. Suggested +testing SSL_read()'s return code, if 0 don't update the screen. + +Created a new data type CK_OFF_T in ckcdeb.h that will eventually resolve +to whatever each platform uses for file sizes and offsets. ckcdeb.h, +17 Nov 2005. + +Made a new library routine ckfstoa() that converts a file size or offset to +a string. This is to solve the problem with having to use different +printf() formats for different representations of file size (int, long, long +long, off_t, signed, unsigned, etc). Replaced a few printf("%l",size) with +printf("%s",ckfstoa(size)) with the expected results. This is just a start, +the definitions will need adjustment for many platforms, variables need to +be redeclared, and all the offending printf's (and printw's) will have to +hunted down and converted. ckclib.[ch], ckuus4.c, 17 Nov 2005. + +Built a minimal version on Linux with: +make linux "KFLAGS=-DNOLOCAL -DNOICP -DNOCSETS -DNODEBUG" +Worked fine, result was 260K on i686. 21 Nov 2005. + +Discovered that Kermit's date parser, contrary to the documentation, failed +to handle strings like "Wed, 13 Feb 2002 17:43:02 -0800 (PST)", which are +commonly found in email. This was because of an overzealous and misguided +check in the code; once removed, all was well. ckucmd.c, 26 Nov 2005. + +Added a new format code 4 to \fcvtdate() to emit asctime() format, used in +BSD-format email message envelopes (i.e. the "From " line). shuffledate(), +ckucmd.c, ckuus[24].c, 26 Nov 2005. + +Added a new function \femailaddress(). Given a From: or Sender: header line +from an RFC2822-format email address, extracts and returns the actual email +address, such as kermit@columbia.edu. ckuusr.h, ckuus[42].c, 26 Nov 2005. + +Using the new functions, I wrote a script to fetch mail from a POP3 server +over a TLS connection. But the line-at-a-time input (needed for changing +line terminators and byte-stuffing text lines that start with "From ") is +slow, 17 sec to read 29 messages totaling 175K. + +Added INPUT /CLEAR so INPUT can be started with a clean buffer without +requiring a separate CLEAR INPUT command. ckuusr.h, ckuus[r24].c, +27 Nov 2005. + +One thing that INPUT was never able to do well was read and save the +complete incoming data stream. That's because, while waiting for its +target, the buffer might overflow wrap around. Yet there was never a way to +tell it to stop when its buffer fills up and let me save it. I added a +/NOWRAP switch that does this. If the buffer fills up before any other +completion criterion is met, INPUT returns failure, but with \v(instatus) +set to 6 (the next available instatus value). Thus a program that wants to +read and save (say) an email message from a POP server, which could be any +length at all, and which terminates with . could do this: + + set flag off + while open connection { + input /nowrap 10 \13\10.\13\10 # Wait for . + if success { + frwrite /string \%o {\freplace(\v(input),\13\10.\13\10,\13\10)} + set flag on + break + } else if ( == \v(instatus) 6 || == \v(instatus) 1 ) { + frwrite /string \%o {\v(input)} + continue + } + break + } + if flag (handle success) + +Note carefully the braces around the FWRITE text; without them, trailing +spaces would be lost. + +Previously the only way to INPUT an entire data stream without losing +anything (assuming it was ordinary lines of text that were not "too long"), +was line-by-line: + + while open connection { + input /clear 10 \13\10 + if fail break + if eq "\v(input)" "$ \13\10" break + fwrite /string \%o {\freplace(\v(input),\13\10,\10)} + } + +The new code is 3 times faster using the default INPUT buffer length of 4K. +Raising it to 16K makes it 3.6 times faster (not worth it). Changing the +POP3 script to use INPUT /NOWRAP makes it about twice as fast (it does more; +it has to do all the byte-stuffing and unstuffing). 27 Nov 2005. + +Changed ssl_display_xxx() to just return if SET QUIET ON. Otherwise there +is no way to suppress the messages. Also protected a previously unprotected +printf("[SSL - OK]\r\n"); by if ( ssl_verbose_flag ). ck_ssl.c, +28 Nov 2005. + +Discovered that FOPEN /APPEND doesn't work if the file doesn't exist. It +uses cmiofi() which is a super-hokey front end to cmifi2(). I had code to +call it but for some reason it was commented out, with a note to the effect +it didn't work. I uncommented it but that didn't help much. So I wrote an +entirely new cmiofi() that works exactly as it should, using chained FDBs, +_CMIFI to _CMOFI (I think the original cmiofi() predated chained FDBs). +ckuus7.c, ckucmd.c, 29 Nov 2005. + +Getting rid of the awful hacks required to call cmiofi() meant I also had to +change the EDIT command, which is the only other place where it's used. +Unfortunately now it's no longer possible to give EDIT without a filename +(to just start an empty editor) but I doubt anyone will notice. ckuusr.c, +29 Nov 2005. + +IF KERBANG didn't always work right. If a kerbang script TAKEs another +kerbang script, the second one should have IF KERBANG false, but it didn't. +Added a check for \v(cmdlevel) == 1. Now you can write a wrapper that runs +a kerbang script in a loop, and the latter can use IF KERBANG to know +whether to EXIT (if called at top level) or END (if called by another +script, thus allowing -- in this case -- the loop to continue). ckuus6.c, +29 Nov 2005. + +Changed \flop() and flopx() functions to take a third argument, a number +signifying at which occurrence of the break character to lop, so: + + \flopx(sesame.cc.columbia.edu) = edu + \flopx(sesame.cc.columbia.edu,,2) = columbia.edu + +ckuus[24].c, 1 Dec 2005. + +Built OK on VMS 7.2-1 with MultiNet 4.4. Built with and without OpenSSL on +Linux OK, ditto Solaris 9. Built OK on RH Linux AS4 on X86_64 (64-bit); +"show var fsize" (using new ckfstoa()) works OK there. Also Mac OS X 10.3.9 +(32-bit), Tru64 UNIX 4.0F (64-bit), HP-UX 11iv2 (64-bit) (picky new compiler +spews out tons of useless warnings), FreeBSD 6.0 on ia64 (64-bit). + +--- Dev.07 --- + +Changed "make netbsd" to be a synonym for "make netbsd2" because the +original netbsd target was ancient. Renamed it to netbsd-old. makefile, +3 Dec 2005. + +Updated INPUT and MINPUT help text. ckuus2.c, 3 Dec 2005. + +Discovered that on a SET PORT /SSL connection, Kermit treats incoming +0xff data bytes (e.g. sent from the POP server) as IACs and goes into Telnet +negotiations. Jeff says "You will need to implement NP_SSLRAW and NP_TLSRAW +that do the same as NP_TCPRAW but negotiate SSL or TLS as appropriate." +This was not as easy as it sounded, because apparently a lot of the Telnet +code is used by SSL and TLS even when Telnet protocol is not being executed. +I wound up doing this as follows: I added /SSL-RAW and /TLS-RAW to the +switch table. Rather than disable Telnet, they do exactly what the /SSL and +/TLS switches do, but also set a special flag. This flag is checked in only +two place: netclos() (to prevent Kermit from sending TELNET LOGOUT when +closing the connection), and tn_doop() (to prevent Kermit from reacting to +incoming IACs; it makes tn_doop() return(3), which means "quoted IAC", which +causes the caller to keep the IAC as data). ckcnet.h, ckctel.h, ckctel.c, +ckuus7.c, 4 Dec 2005. + +The INPUT command did not account for tn_doop() returning 3. Fixed in +doinput(), ckuus4.c, 4 Dec 2005. + +Added another debug() statement in FTP secure_getbyte() to see what's going on +with Muhamad Taufiq Tajuddin's 205-byte-per-second FTP/SSL downloads, plus +new code to test SSL_read()'s return code (byte count); if 0 don't update +the screen. ckcftp.c, 4 Dec 2005. + +--- Dev.08 --- + +Fixed a typo in the non-ANSIC definition of ckfstoa(). ckclib.c, 7 Dec 2005. + +Our Ctrl-C trap (the ON_CTRLC macro) wasn't working for kerbang files. +Rearranged some code to make it work. ckcmai.c, 8 Dec 2005. + +Started converting code to use CK_OFF_T for file sizes and offsets, and +all [s]printf's to replace "%ld" or whatever with "%s", and the size +variable with a call to ckfstoa(). Since I haven't actually changed the +definition of CK_OFF_T from what all the size variables were to begin +with (i.e. long), it shouldn't do any harm. So far just ckcfn3.c +10 Dec 2005. + +An updated HP-UX 9.xx makefile target from PeterE to fix a core dump that +happens on that platform due to insufficient resources. 14 Dec 2005. + +Added debug() statements to http_blah() routines to tell whether the +connection is "chunked". There seems to be a bad performance problem. +ckcnet.c, 14 Dec 2005. + +PeterE complained about ugly DIRECTORY error message, ?No files match - +"{blah}". The braces are used internally in case the user typed more than +one filespec. I changed the error message to remove them. Ditto DELETE. +ckuus6.c, 15 Dec 2005. + +The problem with HTTP downloads is that Kermit always does single-character +read() or socket_read() calls (or the SSL equivalent); see http_inc(). I +added buffering code for non-SSL connections only but it's gross because it +has to swap ttyfd and httpfd before calling nettchk(). I tried making a +nettchk() clone that accepts a file descriptor as an argument but it didn't +work because too many other routines that are invoked directly or implicitly +by nettchk() (such as in_chk()) are still hardwired to use ttyfd. HTTP GETs +are now 20 times faster on the local network (the improvement is less +dramatic over a clogged Internet). ckcnet.[ch], 15 Dec 2005. + +--- Dev.09 --- + +HTTP file-descriptor swapping is not thread safe. Doing it right, of +course, is a big deal, so for now I just don't define HTTP_BUFFERING for +Windows. ckcnet.c, 15 Dec 2005. + +Noticed that HTTP not included in FreeBSD and OpenBSD builds. Fixed in +ckcdeb.h, 22 Dec 2005. + +Fleshed out 32/64-bit data type definitions and changed struct zattr +(file attribute structure) members length and lengthk to have the new +CK_OFF_T type. Changed final arguments of debug() and tlog() to be the new +LONGLONG type. ckcdeb.h, 22 Dec 2005. + +Changed ckfstoa() to return a signed number in string form, rather than an +unsigned one. That's because off_t is signed (thank goodness). Added the +inverse function, ckatofs() so we can convert file sizes and offsets back +and forth between binary number and string. ckclib.c, 22 Dec 2005. + +Changed Attribute Packet reader to convert incoming file size attribute +with ckatofs() rather than atol(). ckcfn3.c, 22 Dec 2005. + +Converted debug(), tlog(), ckscreen(), etc, to handle potentially "long long" +arguments by making their "n" argument CK_OFF_T. ckuusx.c, ckcdeb.h, +22 Dec 2005. + +Converted the rest of the source files to use CK_OFF_T for all file size +and offset and byte-count related variables, and converted all references to +these variables in printfs to go through ckfstoa(). Then I built it on +Linux/i386 with: + + make linux "KFLAGS=-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64" + +which makes off_t be 64 bits and magically makes all the regular file APIs +use 64-bit sizes and offsets without changing the API calls in the source +code. It's going to be a lot of work to get through all the kinks but I was +able to send a long file, do directory listings of long files, do +\fsize(longfile), etc. When it sends a file, the length is shown correctly +in the A packet. If the receiver does not support big numbers, it receives +the file OK anyway, without showing the size, the thermometer, or percent +done (and then will get an error when the file keeps coming after the 2G +mark). Kermit 95 actually refuses long files for "Size", but only if the +announced is less than 2^63 bytes. When today's Linux version receives a +file, it shows the length correctly in the file-transfer display, as well as +percent done, thermometer, etc. Also built this version on true 64-bit +Linux, and it worked fine. Many files changed, 22 Dec 2005. + +For the record, this API is specified in X/Open's Single UNIX Specification +Version 2, which is branded as UNIX 98. It is called Large File Support, or +LFS, and was developed at the Large File Summit. + +It looks like the operative feature-test macro in glibc for transitional +large file support is __USE_LARGEFILE64. So if this is defined, we can also +supply _LARGEFILE_SOURCE and _FILE_OFFSET_BITS=64 automatically for 32-bit +Linux builds. But there's a Catch-22, you don't know if this is defined +until you read the header files, but you have to define _LARGEFILE_SOURCE +and _FILE_OFFSET_BITS before you read the header files. Maybe it's good +enough to grep through for __USE_LARGEFILE64. makefile, +23 Dec 2005. + +Checked this on true 64-bit Linux. The same symbols are defined in CFLAGS, +but they do no harm; it builds without complaint and works fine. 24 Dec 2005. + +Built it on Red Hat Linux 6.1 from 1999. This picked up the long file +support too. Guess 6.1 isn't old enough to not have it! Kermit seems to +work OK on regular files but I don't have enough disk space to create a long +file, and my bigfile.c program (which creates a long file containing only 1 +byte) doesn't work ("fseeko: invalid argument"). It looks like parts of +this API were visible in Linux before they were actually working. +24 Dec 2005. + +Converted all fseek() and ftell() to macros that expand to fseek() and ftell() +or fseeko() and ftello() depending on whether _LARGEFILE_SOURCE is defined. +ckufio.c, ckuus7.c, ckuusx.c, 24 Dec 2005. + +Made a CK_OFF_T version of cmnum(). It would be a very big deal to just +change cmnum() to return a new type, so another idea is to rename cmnum() to +something else, cmnumw(), change its result argument to CK_OFF_T, and then +make a stub cmnum() to call it to get an int, then call cmnumw() explicitly +any time we need a big number. ckucmd.c, 24 Dec 2005. + +Calling cmnumw() directly requires changes to each routine that uses it. +The INCREMENT and DECREMENT commands, for example, required changes to +doincr(), varval(), and incvar(), and all references to them. ckuusr.[ch], +ckuus[56].c, 24 Dec 2005. + +Calling cmnumw() in chained FDBs required defining a new function code, +_CMNUW, adding a new member to the OFDB struct for returning wide results, +and adding a new case to cmfdb(). ckucmd.[ch], 24 Dec 2005. + +Changed FSEEK and FCOUNT to use the new chained FDB interface, now we can +seek and look past 2GB. ckuus7.c, 24 Dec 2005. + +Next come switches, which store their results in a struct stringint. This +struct was defined in each module where it was used (ckuus[r367].c, ckcftp.c). +I moved the definition to ckuusr.h and added a wval member, which can be +referenced by any switch-parsing code that calls cmnumw(). 24 Dec 2005. + +Changed SEND /CALIBRATE:n to allow big values of n. This makes it possible +to test the protocol aspects of long-file transfer without actually having a +long file handy. ckuusr.c, 24 Dec 2005. + +SEND /SMALLER-THAN:n, SEND /LARGER-THAN:n, and and SEND /START:n also now +allow large values of n. ckuusr.c, 24 Dec 2005. + +Changed the algebraic expression evaluator to use wide values. +ckuus5.c, 24 Dec 2005. + +Fixed ckfstoa() to handle the case when n is negative and (0 - n) is also +negative, which happens for numbers 2^(n-1) or greater, where n is the +number of bits in the word size we're dealing with, e.g. 64, in which case +2^63 has its sign bit set so seems to be negative. In such cases, ckfstoa() +returns "OVERFLOW" instead of a numeric string. We'll have to see how this +plays out but I think it's better to cause a parse error and stop things +dead than to return a spurious number. ckclib.c, 24 Dec 2005. + +Converted the S-Expression handler to use wide integers. ckuus3.c, 24 Dec 2005. + +Took all the LONGLONG stuff out of ckcdeb.h, we don't need it. + +All of these changes result in 64-bit arithmetic (more or less) on 32-bit +Linux, as well as on true 64-bit platforms. + +Rebuilt today's code on Solaris 9 in the 32-bit and 64-bit worlds, on Red +Hat 6.1, Red Hat AS4.2. I haven't bothered trying a 32/64 hybrid build for +Solaris, since I can build a pure 64-bit version there. Quick tests show +the large-number arithmetic works OK in all cases except, of course, on pure +32-bit builds (unfortunately I can't find a running Linux system old enough +to verify this for Linux, but it's true for other 32-bit platforms). +24 Dec 2005. + +Tried building a hybrid version on Solaris 9 after all since the LFS API is +ostensibly the same as for Linux: + + make solaris9 "KFLAGS=-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64" + +It built smoothly and the resulting binary is 2.5MB compared to 3.4MB for +the 100% 64-bit version. Looks like a keeper. For now, added solaris9lfs +and solaris10lfs entries to the makefile but if these work on PCs we can +make these the regular entries for Solaris 9 and 10. 27 Dec 2005. + +Built on Mac OS X 10.4 with the regular target. It seems that in that case, +off_t is 64 bits anyway. Noticed that a lot of stuff didn't work, like +exponentiation in S-Expressions. Tried building it as above, which worked, +and now CK_OFF_T is 64 bits instead of 32, but (^ 2 30) is still 2.0. In +fact 2-to-the-any-power is 2.0. It seems that the Mac OS X version did not +have FNFLOAT defined. It also seems that every test in dosexp() like: + + if (result != fpresult) fpflag++; + +should have been protected by #ifdef FNFLOAT..#endif /* FNFLOAT */ -- a +double-ended break, as they say in the nuclear power industry. ckuus3.c, +27 Dec 2005. + +Added GREP /EXCEPT:pattern. ckuus[26].c, 27 Dec 2005. + +Fixed a problem with uninitialized pv[].wval (switch-parsing parameter-value) +members that showed up on certain platforms or with certain compilers. Now +the Mac OS X 10.4 version works. ckuus[r367].c, ckcftp.c, 28 Dec 2005. + +Built on Unixware 7.1.1, a pure 32-bit build, seems fine. Rebuilt on Red +Hat AS 4.2 just to make sure I didn't break anything, it's OK. No testing +on HP-UX, etc, because HP testdrive file system is full, can't upload +anything. 29 Dec 2005. + +Commented out the SHOW FEATURES section that displays constants like +INT_MAX, CHAR_MAX, etc, because printing each value in the appropriate +format is too tricky, and we don't need them anyway. ckuus5.c, 29 Dec 2005. + +Updated ckvfio.c to use CK_OFF_T for the relevant variables. Built and +tested on VMS/Alpha 7.2: file transfer in remote mode; making a Telnet +connection and then local-mode file transfer; S-Expressions, all OK. Also +built a no-net version OK. 29 Dec 2005. + +Built and tested on Red Hat AS4 AMD X86_64, used it to upload new sources to +FreeBSD 4.11. Built on FreeBSD 4.11/i386. Here's another one where off_t +is 64 bits, even though long is 32 bits. But it seems to work ok, not sure +why, when CK_OFF_T is 32 bits. There is no _LARGEFILE_SOURCE stuff in the +header files. 29 Dec 2005. + +Built on Mac OS X 10.3.9 using the new macosx10.4 target to pick up LFS. +Works fine. + +Built on Red Hat Linux 4WS on IA64 (64-bit). Now this one is odd, stat() +fails on big files. It happens also if I use the "linuxnolfs" target, which +does not define _USE_LARGEFILE or _FILE_OFFSET_BITS=64. DIRECTORY BIGFILE +shows the size as -1, but if "log debug", it says "no files match", i.e. +different behavior, observer effect. I hate when that happens. + +Let's see if that's an anomaly... Built on Tru64 Unix 4.0F (64-bit Alpha). +It sees long files just fine. Rebuilt and checked on x86_64 again... fine. +OK, let's not worry about IA64 yet. + +Another small fix to the HP-UX 9.0 target from PeterE. makefile, 29 Dec 2005. + +---Dev.10--- + +Code adjustments from Jeff, mainly to the SSL and TLS Raw mode code from +several weeks ago, plus changing some data types in the security code to +CK_OFF_T, plus a different data type for CK_OFF_T for K95 because Windows +size_T isn't signed. This presumably will allow large-number arithmetic but +it will not give large file access because that will require replacing all C +library file i/o calls (esp. in ckofio.c) with native Windows APIs. Build +on Solaris 9 with and without SSL and on Linux RH AS4.2 with and without +SSL. ck_crp.c, ck_ssl.c, ck_ssl.h, ckcdeb.h, ckcftp.c, ckcmai.c, ckcnet.c, +ckcnet.h, ckctel.c, ckuat2.h, ckuus4.c, ckuus7.c, ckuusr.c, 30 Dec 2005. + +It was reported that WRITE SESSION always returned a failure status, even +when it succeeded. The problem was that Unix versions of zsout() and +zsoutl(), for the session log only, were using write() and returning +write()'s return code, which is different from what zsout() and zsoutl() are +documented to return. Also plugged a couple potential holes in zsoutx() +that I noticed while I was in the neighborhood. ckufio.c, 30 Dec 2005. + +Added FSEEK /FIND:pattern. This form of FSEEK accepts all the other +switches and arguments and performs the desired seek. Then, if the seek was +successful, it starts from that point and reads through the file, line by +line, searching for the first line that contains the given string or matches +the given (unanchored) pattern and, if found, sets the file pointer to the +beginning of that line. Useful, e.g., for very long timestamped logs, where +you want to start processing at a certain date or time; searching for a +particular string is much faster than doing date comparisons on each line. +ckuus[27].c, 30 Dec 2005. + +It was annoying me that FILE STATUS (FSTATUS) required a channel number to +be given even if only one file was open, so I supplied the correct default +in that case. ckuus7.c, 30 Dec 2005. + +INPUT /NOWRAP, added recently, is used for efficiently copying the INPUT +stream intact, but it's not good for matching because if the INPUT target is +broken between the end of the previous buffer and the beginning of the next +one, the context is lost and the match does not occur. I thought of several +ways around this, but they all involve saving a huge amount of context -- +old input buffers, the arrays of target strings and corresponding match +positions, etc. The alternative is fairly simple but it's not transparent +to the user. Here's what I did in a POP script: + + .eom := "\13\10.\13\10" + set flag off # FLAG ON = success + while ( open connection && not flag ) { + .oldinput := \fright(\v(input),8) # Save tail of previous INPUT buffer + input /clear /nowrap 4 \m(eom) # Get new INPUT buffer + if success { # INPUT matched - good + .s := {\freplace(\v(input),\m(eom),\13\10)} + set flag on + } else { # No match + .s := \v(input) # Check if target crossed the border + .oldinput := \m(oldinput)\fsubstr(\v(input),1,8) + if \findex(\m(eom),\m(oldinput)) set flag on + } + ... + } + +I think this will be easier to explain than any dangerous and grotesque +magic I might put into doinput() itself. For now, added a few words about +this to HELP INPUT. ckuus2.c, 30 Dec 2005. + +Back to the pattern matcher. Noticed that "IF MATCH index.html [a-hj-z]*" +succeeded when it should have failed. In ckmatch(), the clist section +needed one more clause: it can't float the pattern if an asterisk does not +occur in the pattern before the clist. This change fixes the problem +without breaking any other cases that weren't already broken, most of which +involve slists, i.e. {string,string,string,...}. ckclib.c, 30 Dec 2005. + +Tried FSEEK /FIND: on a largish file (over 100,000 lines), using it to seek +to a line near the end. It took 0.756 seconds, compared with Unix grep, +which did the same thing in 0.151 sec. That's because C-Kermit is using +ckmatch(). But if the search target is not a pattern, it should be a bit +faster to use ckindex(). Yup, 0.554 sec, a 36% improvement. Can't expect +to compete with grep, though; it's highly tuned for its single purpose. +ckclib.[ch], ckuus7.c, 1 Jan 2006. + +Updated visible copyright dates to 2006: ckcmai.c, ckuus2.c, ckuus5.c, +1 Jan 2006. + +Noticed that NetBSD 2.0.3 has 64-bit off_t, and that _LARGEFILE_SOURCE is +mentioned in . Tried building Kermit with _LARGEFILE_SOURCE added +to CFLAGS, it's good. Added it to the netbsd target. makefile, 1 Jan 2006. + +Fixed typo, #ifdef CK_NOLONGLONG in ckuus5.c should have been #ifndef +CK_LONGLONG (which, it turns out, we don't use anyway). 2 Jan 2005. + +Observed that FreeBSD 4.x has a 64-bit off_t, but does not use the +_LARGEFILE_SOURCE convention. Reasoning that all versions of FreeBSD have +off_t (I was able to check back to FreeBSD 3.3), I simply #define CK_OFF_T +to be off_t in ckcdeb.h within #ifdef __FreeBSD__ .. #endif. Another one +down. This can be done for any platform that is guaranteed to have off_t. +Turns out FreeBSD 3.3 has 64-bit off_t too. 2 Jan 2005. + +OpenBSD, same as FreeBSD. Also, added OS-version-getting thing to makefile +target for the program herald, as in the other BSDs. Built on OpenBSD 2.5 +from 1998, it has 64-bit off_t too. ckcdeb.h, makefile, 2 Jan 2005. + +Dumping the command stack every time there's an error is really too much. +I added SET COMMAND ERROR-DISPLAY {0,1,2,3} to set the verbosity level of +error messages. Only level 3 dumps the stack. ckuus[235].c, 2 Jan 2005. + +Built on HP-UX 11.11 with _LARGEFILE_SOURCE and _FILE_OFFSET_BITS=64. The +result works fine as far as I can tell. It sees big files, it can open +them, seek to positions past the 2^31 boundary. It can send large files. +It can do large-number arithmetic (^ 2 62). The only problem is that during +compilation, every single modules warns: + + cc: "/usr/include/sys/socket.h", line 504: warning 562: Redeclaration of + "sendfile" with a different storage class specifier: "sendfile" will have + internal linkage. + cc: "/usr/include/sys/socket.h", line 505: warning 562: Redeclaration of + "sendpath" with a different storage class specifier: "sendpath" will have + internal linkage. + +These warnings should be perfectly harmless since they are not coming from +C-Kermit code, nor does C-Kermit use either one of those functions. These +warnings don't come out in HP-UX 11i v2, but on that one we get tons and tons +of picky compiler warnings (variables set but not used, defined but not +referenced, etc). A couple, however, turned out to be valid; one case of +"expression has no effect", and two of "string format incompatible with +data type" (I missed a couple file-size printfs). + +There were also numerous warnings about signedness mismatch or sign +conversion of constants like IAC (0xff). Does the HP-UX Optimizing Compiler +have a compiler flag to make all chars unsigned? Yes, +uc, but the man page +says "Be careful when using this option. Your application may have problems +interfacing with HP-UX system libraries and other libraries that do not use +this option". Sigh, better not use it. + +After reviewing "HP-UX Large Files White Paper Version 1.4" and HP's +"Writing Portable Code" documents, I added -D_LARGEFILE_SOURCE +-D_FILE_OFFSET_BITS=64 to the hpux1000 target, which is the basis for all +HP-UX 10.00 and later builds. Large files are available in HP-UX 10.20 and +later. 10.00 and 10.10 were not real releases, and anyway these flags +should be harmless there unless the large-file implementation was only +partly done. Built OK on both PA-RISC and IA64, optimized and plain. +makefile, 4 Jan 2006. + +Built on FreeBSD 6.0 on IA64. All OK except I got a warning about the +argument passed to time() in logwtmp() in ckufio.c. This section had +already been partially fixed; thus I put the improved version into +#ifdef CK_64BIT, which is our newly available symbol that should be +automatically defined for any true 64-bit build. ckufio.c, 4 Jan 2006. + +Finally got around to testing Jeff's changes to SSL/TLS RAW mode from +December 30th against our POP server. It didn't work, couldn't log in. +Tried backing off the ckctel.c changes first; that allowed login and +communication, but it did not suppress activation of Telnet protocol +whenever a 0xff byte arrived. Backed off the rest of the changes and now +all is OK again. ckctel.c, ckcnet.c, ckuus7.c, 9 Jan 2006. + +Built on NetBSD 1.4.1 (1999), found that it did not like the large file +assumption -- fseeko() and ftello() do not exist; added a clause to the +netbsd target to check for fseeko and not define _LARGEFILE_SOURCE if not +found. Oddly enough, off_t is 64 bits anyway, but it doesn't look like the +APIs are half-done. For example, stat() uses off_t (64 bits) for the file +length, but fseek() uses long (32 bits) and there is no 64-bit analog. +Anyway the new netbsd target works on both 1.4.1 and 1.5.2 (no large files) +and on 2.0.3 (large files). makefile, 9 Jan 2006. + +Built on QNX-32 4.25, which has no large file support. Got a few strange +compiler (WatCom) warnings, but it built and runs OK. Noticed that file +transfers into QNX over a Telnet connection can't use streaming, but that's +nothing new to this version; same thing happens with C-Kermit 7.0. 9 Jan 2006. + +Built on IRIX 6.5. I didn't bother with large files there because it does +not support the _LARGEFILE_SOURCE interface; you have to change all the APIs +at the source level from blah() to blah64(). Seems to work fine as a 32-bit +app even though its off_t is 64 bits. Tried a pure 64-bit IRIX 6.5 build +but it dies in ckcnet.c when it hits SOCKOPT_T and GSOCKNAME_T with "The +identifier 'socklen_t' is undefined". + +Looks like I no longer have access to SCO OSR5. + +Made a pure 32-bit build on SCO UnixWare 7.1.4, all OK. Found that this +version also supports LFS, added it to the uw7 target. makefile, 9 Jan 2006. + +--- Dev.11 --- + +Evidently the HP-UX bundled (non-ANSI non-optimizing) compiler doesn't like +long integers in switch expressions. Changed three examples of these in the +S-expression code. ckuus3.c, 10 Jan 2006. + +A section of tstats() where GFTIMER isn't defined (e.g. on Motorola +sv68r3v6) was garbled. Fixed in ckcfn2.c, 10 Jan 2006. + +A fix for setting 921600 bps on Linux from Paul Fulghum, Microgate Systems Ltd. +ttgspd(): ckutio.c, 11 Jan 2006. + +Noticed that when I changed the compact substring notation code back on +August 9th, I broke the ability to use arithmetic expressions within the +brackets, which explains some rather odd behavior I saw with some of my +scripts. Looking more deeply into this, I also see that all the parsers I +have been using up to now for this, as well as for array bounds pairs, have +been inadequate because they never allowed for nested constructions, such as +a member of a bounds pair that itself was an array element, possibly with +another array element as a subscript. I wrote a new routine for this, +called boundspair(), which is like arraybounds() except it accepts an extra +argument, an array of characters that can serve as bounds-pair delimiters, +and it returns the pair separator that was encountered in another new +argument. For the alternative substring notation for [startpos-endpos] I +had to change the delimiter from '-' to '_' because '-' can be used in +arithmetic but '_' is not a recognized operator. This is so I can parse, +e.g. [a:b] or [a_b] in the same context, and then find out which form was +used, e.g. \s(line[9:12]) or \s(line[9_12]); the first string is 4 bytes +long, the second is 12. Everything seems to be OK now. \s(line[10]) gives +everything starting at 10, but \s(line[10:0]) gives the null string. Bad +syntax in the bounds pairs results in a null string; missing pieces of the +bounds pair result in defaults that should be compatible with previous +behavior. ckuus[45].c, ckuusr.h, 13 Jan 2005. + +Changed arraybounds() to call boundspair(). This was a rather drastic +change, not strictly necessary, but I think I got all the kinks out. +ckuus5.c, 13 Jan 2005. + +Changes from PeterE to the makefile for HP-UX 6 and 7, to accommodate bigger +symbol tables, etc. 19 Jan 2005. + +Determined that SCO OSR5.0.6 (and earlier) do(es) not support large files. +Don't know about 5.0.7. 30 Jan 2005. + +Created a new build target for SCO OSR6.0.0. Gets the exact 6.x.x version +dynamically. Supports large files and big-number arithmetic via CK_OFF_T. +The sockopt() family of functions changed the data types of some of their +arguments since OSR5. It was already possible to define SOCKOPT_T and +GSOCKNAME_T from the command line but I had to add code to also allow this +for GPEERNAME_T too. ckcnet.c, makefile, 30 Jan 2005. + +Apparently, ever since C-Kermit 7.0 was released, it has never been possible +to use a variable for the as-name in a RECEIVE command in Kermit 95. This +is because evaluation of the as-name field was deferred until after we could +check whether it might be a directory name (which, in Windows, could start +with a backslash). This little bit of magic was not a good idea, magic +hardly ever is. I changed the code to evaluate both as-name fields in the +normal way. If they want to receive to a directory called "\%1", they'll +just have to spell it differently. The workaround is to turn the whole +command into a macro and evaluate it before executing it, e.g.: + + assign xx receive /as-name:\%1 + do xx + +ckuus6.c, 1 Feb 2006. + +Built OK on FreeBSD 6.1 on AMD64. Adjusted some copyrights and date stamps. +ckcmai.c, makefile, 8 Feb 2006. + +--- Dev.12 --- + +Fixed a signed/unsigned char warning in the new boundspair() calling code +in the compact substring notation handler. ckuus4.c, 9 Feb 2006. + +Removed a spurious extra linux+openssl label from the makefile, added +solaris10g_64 synonym. 9 Feb 2006. + +Satisfied myself that LFS is OK on Solaris 10 i386, and I'm going to assume +it's also OK on Solaris 9. Made LFS standard for all Solaris 9 and 10 +builds (including the secure ones) except the explicitly 64-bit ones, and +made the provisional solarisXXlfs targets into synonyms. makefile, 9 Feb 2006. + +--- Dev.13 --- + +Further attempts at SSL/TLS message suppression when QUIET is ON. +ck_ssl.c, 16 Feb 2006. + +From J.Scott Kasten: (quote...) I just uploaded a patch to /kermit/incoming. +The file name is "jsk-patch-for-cku211.diff". I have also included the +patch as ASCII text in this email below. This patch may be applied to the +cku211.tar.gz source code via: + cd cku211, patch -p1 <../jsk-patch-for-cku211.diff +The patch adds 4 new build targets: + netbsdwoc - a stripped no curses target for iksd used. + netbsdse - security enhanced target with srp, ssl, and zlib. + irix65gcc - build on SGI Irix 6.5 platform using gcc. + irix65se - security enhanced target with srp, ssl, and zlib. +The patch fixes one build target: + irix64gcc - The "-s" option is not supported by gcc under Irix. +I thank all of you in the Kermit Project for such a fine utility. I +recently had to get a 16 MB file overseas across a spotty communications +link to repair a computer remotely. Kermit was the only thing that could do +the job, so I wanted to contribute these patches back to the mainstream to +say thanks. This digitally signed email is a binding contract that +officially assigns the rights to the source code patch (shown below) that I +developed to the Kermit Project at Columbia University. (...end quote) +ck_ssl.c, makefile, 23 Feb 2006. + +Changed the new NetBSD target names to be consistent with the conventions +used in most other targets: + + netbsdwoc -> netbsdnc + netbsdse -> netbsd+ssl+srp+zlib + irix65se -> irix65+ssl+srp+zlib + +and removed old, now superfluous, NetBSD targets (old-netbsd, netbst15, +netbst16), leaving synonym labels in their place. Also updated (crudely) +the Linux target variations (curses instead of nocurses, no curses at all) +to be (appropriately modified) copies of the current linux target. It would +be nicer to combine them, but this gets the job done. makefile, 23 Feb 2006. + +--- Dev.14 --- + +Fixed the HELP command when used with tokens like @, ^, #, and ;. The first +two had been omitted from the table. The second two required a new path +into the guts of the parser, since comments are normally stripped at a very +low level. ckuus[r2].c, ckucmd.c, 24 Feb 2006. + +Built on AIX 5.1 ("make aix51") without incident. Then I tried: + + make aix51 "KFLAGS=-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64" + +This had no effect. I found the relevant document ath the IBM website. It +says to use -D_LARGE_FILES instead. I added this to the AIX 4.2 target +since (a) IBM says large files are supported by AIX 4.2 and later, and (b) +all Kermit AIX targets past 4.2 use the 4.2 one. Plus a clause to make +sure CK_OFF_T is defined appropriately. ckcdeb.h, makefile, 6 Mar 2006. + +Added a 32-bit aix51+openssl target. Builds OK, works fine (tested against +our SSL POP server). Tried I tried adding -D_LARGE_FILES. It seems to work +fine, so we'll keep it. Cleaned up the other aix5blah entries a bit also. +makefile, 6 Mar 2006. + +Fixes from J. Scott Kasten to the IRIX 6.4 and 6.5 makefile targets. They +were badly wrong. makefile, 6 Mar 2006. + +The reason Kermit was looping on directories in IRIX was a classic +"double-ended break". The makefile targets failed to define DIRENT so +Kermit was open/read on directories rather than opendir()/readdir(). But +then it was also failing to account for the fact that read() would return -1 +on error. The makefile fix adds -DDIRENT, and the read() case in traverse() +now properly terminates its loop on error. ckufio.c, 6 Mar 2006. + +--- Dev.14 --- + +In response to a complaint that C-Kermit would not build on HP-UX 11 with +OpenSSL, I tried it myself on both 11.11/PA-RISC and 11i v2/Itanium. It built +OK on both but I had to add a new target (hpux1000o+openssl-nozlib) for no +Zlib since these boxes did not have it installed. makefile, 9 Mar 2006. + +Added OpenSSL version number display to SHOW FEATURES. ckuus5.c, 9 Mar 2006. + +Gavin Graham noticed that FTP [M]GET /DELETE /MOVE-TO: was rejected with +"?Sorry, /DELETE conflicts with /MOVE or /RENAME". This check belongs in the +PUT code but not in the GET code. Commented it out and tested the result. +The combination is now accepted but then Kermit refuses the incoming file as +if it had been given a /SMALLER-THAN: or /LARGER-THAN: switch, which it didn't +happen. Turns out there was one more place where I wasn't initializing the +new "wide int" member of the switch-parsing pv[] struct. Once this was fixed, +the /MOVE-TO part still didn't work. Turned out the /DELETE case was part of +a long if-else-if-else- chain, which effectively made /DELETE and /MOVE-TO: or +/RENAME-TO: mutually exclusive. Fixed this, now it works fine. ckcftp.c, +13 Mar 2006. + +Got access to AIX 5.3, built there, all OK, including large files. 13 Mar 2006. + +--- Dev.16 --- + +Patches from Mark Sapiro to suppress getsockopt() and getsockname() warnings +in Mac OS X. ckcnet.[ch], 18 Mar 2006. + +In response to a complaint from Clarence Dold, tried "make redhat9" (which +is the rather dated target that tried to include all forms of security) on +RH Linux AS4.3, it failed miserably. I made a new makefile target, removing +Kerberos IV and got a lot farther. But then in ckcftp.c, the following +struct definition: + + struct { + CONST gss_OID_desc * CONST * mech_type; + char *service_name; + } gss_trials[] = { + { &ck_gss_mech_krb5, "ftp" }, + { &ck_gss_mech_krb5, "host" }, + }; + +refers to a variable, ck_gss_mech_krb5, that is not defined anywhere. Up +above, however, is a static definition for gss_mech_krb5, so I changed the +struct definition to match. Next, in ckuath.c, the compiler could not find +the com_err.h file. Turns out in Linux this is in a subdirectory, et, so we +have to add a -I clause to the makefile target for this. Made a target for +Linux+SSL only. Made a target for Linux+Krb5 only; this required moving an +#ifdef in ckuus7.c to prevent an unguarded reference to SSLEAY_VERSION. +New targets: linux+krb5+ssl, linux+krb5, linux+krb5. ckcftp.c, ckuus7.c, +makefile, 27 Mar 2006. + +New targets of HP-UX 10/11 with OpenSSL from PeterE. makefile, 27 Mar 2006. + +Added large file/integer support to SHOW FEATURES. ckuus5.c, 27 Mar 2006. + +Built OK on Solaris 9 and 10 with gcc (someone was complaining that this +didn't work, but that was 8.0.211). + +Started build on a Sun 3/80 mc68030 with NetBSD 2.0 and gcc 3.3.3. But it +died with an assembler error in ckcfn2.c (compiler bug). 27 Mar 2006. + +--- Dev.17 --- + +NebBSD 2.0 build completed by turning off optimization on ckcfn2.c +("KFLAGS=-O0"). Result supports 64-bit ints and, presumably, large files. +uname -p = "m68k", -m = "sun3". 29 Mar 2006. + +Corrected an omission in applying PeterE's updates to the HP-UX targets. +makefile, 28 Mar 2006. + +solaris2xg+krb5+krb4+openssl+shadow: + +Tried resurrecting the solaris2xg+krb5+krb4+openssl+shadow target. It asks +to link with libdes but there is no libdes. Removed -ldes from the target, +now at least it builds and runs wart. The compilation blows up in ckcftp.c +for missing header files: + + ckcftp.c:462: kerberosIV/krb.h: No such file or directory + ckcftp.c:500: gssapi/gssapi_generic.h: No such file or directory + ckcftp.c:501: gssapi/gssapi_krb5.h: No such file or directory + +Got a bit farther by adding appropriate -I's and -L's to KFLAGS but it still +dies compiling (or linking?) ckcftp.c, but it doesn't say exactly why. OK, +deferred. + +Added SET SEXPRESSION TRUNCATE-ALL-RESULTS { ON, OFF }. This can be used +for force integer arithmetic in any kind of calculation that requires it, +such as date calculations. This is a global setting, not on any kind of +stack. Also, updated SHOW SEXP and added HELP SET SEXP which wasn't there +before. ckuus[23].c, 30 Mar 2006. + +To make the RENAME command a bit more useful, need to add some switches. +But it shares a switch table, qvswtab[], with some other commands. Broke +this off into its own switch table. ckuus6.c, 17 Apr 2006. + +Added RENAME switch values that can be used in the same table with the DELETE +switch values, which are shared by many commands. ckuusr.h, 17 Apr 2006. + +Discovered that the RENAME command could be entered without any arguments +and it would still succeed. Fixed in dorenam(): ckuus6.c, 17 Apr 2006. + +Added parsing for RENAME /UPPER:option (to uppercase the file name(s)), +/LOWER:option (to lowercase), and /REPLACE:{{s1}{s2}} (to do string +replacement on the filename(s)), but not the semantics. When any of these +switches is given, the target ("to") name is not parsed; they act on the +source name. The /LOWER: switch takes keyword args to specify whether it +should act only only files that have all UPPER case latters, or on ALL files +(i.e., including files with mixed-case names); similarly for the /UPPER: +switch. There is some creative parsing allowing these to be given with or +without a colon and keyword argument, which works fine except if you include +the colon but no argument, execute the command (which works fine), and then +recall the command. I haven't yet decided about the interaction among these +switches. Clearly if /UPPER is given after /LOWER, it overrides. But if +/UPPER (or /LOWER) is given with /REPLACE, what should happen? ckuus6.c, +17 Apr 2006. + +Filled in actions for RENAME /UPPER: and /LOWER: for the single file case, +and tested all combinations of switch values and filename configurations. +Once that was OK, moved the code out into a separate routine, renameone(), +and then called it from both the single-file case and the multifile case. +ckuus6.c, 19 Apr 2006. + +Added RENAME /SIMULATE. Filled in the code for string replacement, needs +testing. ckuus6.c, 20 Apr 2006. + +Changed /REPLACE options to allow a negative number to specify an occurrence +from the right, so -1 means the last occurrence, -2 means the next-to-last, +etc. ckuus6.c, 24 Apr 2006. + +Added RENAME /COLLISION:{OVERWRITE,PROCEED,FAIL}. This is implemented but +not tested. ckuus6.c, 24 Apr 2006. + +Worked on RENAME /COLLISION:FAIL. I decided it was less than useful to ... + +Added SET RENAME { COLLISION, LIST } to let user change default collision +and listing actions. ckuusr.[ch], ckuus[36].c, 25 Apr 2006. + +Experimented with parsing for /CONVERT:cset1:cset2. The problem here is +that there is no straightforward way for a switch to have multiple +arguments. Or is there...? If I parse cset1 with cmswi() rather than +cmkey(), it almost works; the only problem is that the character-set +keywords don't have CM_ARG set, so they don't know to stop on, and ignore, a +colon. If I make a copy of the table and set CM_ARG in the flags field for +each keyword, it works fine: if I Tab in the first name, it fills itself +out, supplies a colon, and waits for the second name. So in the code, the +first time that RENAME /CONVERT is invoked, I put code to copy fcstab[] and +set CM_ARG in each flags field. Works fine, and now we know how to make a +switch that takes multiple arguments. ckuus6.c, 24 Apr 2006. + +I thought I had a function to convert the character set of a string but I +don't, so actually implementing /CONVERT: will be difficult. + +Actually the parsing wasn't that easy either. It works OK interactively, +but not in a TAKE file. To make a long story short, I had to change +gtword() and cmkey2() to not require "/" at the beginning of a switch, and +then to parse arguments-that-are-followed-by-other-arguments as if they were +switches, so that they can end with colon rather than space. This might +seem dangerous, but switches always have "/" at the beginning, so the check +is superfluous. ckucmd.c, 26 Apr 2006. + +Back to /CONVERT... Once I was able to get the code to call cvtstring() I +was able to debug it (at first it was skipping every second character). And +now we have a general-purpose string-translating function we can call from +anywhere. Requires that C-Kermit be built with Unicode support. +ckuus6.c, 26 Apr 2006. + +Added SHOW RENAME. ckuusr.h, ckuus[r5].c, 26 Apr 2006. + +Conditionalized some Unix/Windows assumptions in renameone() so the code +could work in VMS. ckuus6.c, 2 May 2006. + +Added RENAME /FIXSPACES to change all spaces in the filename(s) to +underscore or any other character or string that is given. This is just a +special case of RENAME /REPLACE:{{ }{x}} with easier syntax. +ckuusr.h, ckuus6.c, 2 May 2006. + +Added an "all-but" control to the /REPLACE options: +/REPLACE:{{.}{_}{~1}} means replace all but the first (this one works); +/REPLACE:{{.}{_}{~-1}} means replace all but the last (this one not yet). +ckuus6.c, 2 May 2006. + +Filled in the second one ("all but" the given occurrence). The algorithm is +simply to reverse the three strings and then use the same code as we use in +the left-right-case, and then unreverse the result. At first I used +yystring() for this but yikes, what a bad design! So I made a better +string-reversal routine, gnirts(), for this (luckily yystring() is only used +in one place, for which its design is appropriate). ckuus6.c, 3-4 May 2006. + +Added code to handle the case where the file being renamed includes a path +specification. In this case we separate the path, apply the renaming +functions to the filename only, and then at the end rejoin the original +filename with the path, and join the new name with same path or, if a +destination directory was given, with that. ckuus6.c, 4 May 2006. + +Added HELP SET RENAME and updated HELP RENAME. ckuus2.c, 4 May 2006. + +"Tom Violin" (Tom Hansen) noticed that the first time you FOPEN a file, +Kermit's memory consumption goes way up. In fact there's a warning to that +effect in the code, where, upon first open, a potentially big array of +potentially big structs is allocated. I rewrote the code to allocate each +array member (struct ckz_file) as needed, i.e. when a file is opened, and to +free it when the file is closed (or the open fails). This was actually +quite a lot of work, which is why I didn't do it the first time around: +every single "." had to be changed to "->". Every check for a valid +channel first had to check if the channel's struct was allocated and every +other reference to z_file[i]->anything had to be prechecked that z_file[i] +was not a NULL pointer. Also I made some improvements to FILE STATUS, and I +fixed FILE CLOSE to default the channel number if only one channel was open, +as I did for FILE STATUS a while back. ckuus7.c, Cinco de mayo 2006. + +Ran my old BUILDS script that builds C-Kermit with about 100 different +combinations of feature-selection switches. Fixed a few small glitches so +now they all build OK (except can't do NOANSI builds any more on recent +Linuxes because of varargs()). ckuus3.c, ckuus5.c, ckuus6.c, ckuus7.c, +ckucmd.c, ckcfns.c, 6 May 2006. + +Fixed RENAME /LOWER and /UPPER, when given with no colon or argument, to +default to ALL. ckuus6.c, 13 May 2006. + +Built on VMS 7.2-1, tested new RENAME command there; seems to be OK. +13 May 2006. + +--- Dev.18 --- + +I wanted to test large files against RESEND but I don't have access to any +system that can run C-Kermit and that also has enough space for a large +file. I created a "fake" large file on Linux (3G hole plus 1 byte), and +sent it over a localhost connection, and interrupted it repeatedly and then +initiated a RESEND at the sender. In each case, it picked up where it left +off. But before the 2G boundary was crossed the disk filled up. +Inconclusive. 14 May 2006. + +PeterE got a warning in the new FILE OPEN code when building in HP-UX 9. +I added a cast, built on HP-UX 11, no more complaint. However there +are warnings about internal vs external bindings of sendpath and sendfile +in every module. Too bad, these are not Kermit tokens, it's a conflict in +HP's header files. Marc Sapiro doesn't see them; probably it's something +on the HP testdrive site. ckuus7.c, 17 May 2006. + +Fixed the tru64-51b+openssl target -- the terminating doublequote of KFLAGS +was missing -- and also the osf target, which failed to import the LIBS +definition from whatever other target invoked it. Now the SSL build goes OK +on Tru64 5.1B. Replaced x.tar.z in the download areas without declaring a +new Dev number. The new one has a makefile with today's date. Software +engineering at its best! makefile, 18 May 2006. + +Scott Kasten noted that the estimated-time-remaining calculation would go +bonkers on LFS systems when RESENDing a large file. It looks like the +shocps() and shoetl() functions escaped the CK_OFF_T conversion. I made +what seemed to be the right adjustments, and then was lucky enough to find a +computer that had enough free disk space for me to send a large file, +interrupt it several times, resend it, all seems to be OK. 28 May 2006. +Later Scott verified these changes independently for Linux, but the problems +in IRIX remain. + +Patches from Scott Kasten for large files on IRIX 6.5: ckcdeb.h, makefile, +12 Jun 2006. + +--- Dev.19 --- + +Added a new function for dealing with JPGs and GIFs: + +\fpicture(filename,&a) + returns 0 if file not recognized or can't be opened; + returns 1 if landscape, 2 if portrait or square. + If array given, element 1 is width, element 2 is height. + +ckuusr.h, ckuus4.c, 19 Jun 2006. + +Scott Kasten reports that the FTP client can transfer large files OK, at +least in Linux, but has trouble with recovery: + + . Kermit takes a very long time to start the transfer, sometimes over + 30 minutes. Suspect the ftp server is counting the bytes in a long file? + Or maybe it's a text-mode transfer and it's counting the lines? Probably + in response to Kermit's SIZE command. + + . The size shown in the FT display is wrong by a random amount. And of + course so are the progress bar, percent done, and time remaining. + + . The file, however, is transferred correctly. REGET works correctly too. + +I tried setting up a test scenario locally but our Solaris FTP server does +not support large files: + + FTP SENT [SIZE BIGFILE] + FTP RCVD [550 BIGFILE: not a plain file.] + FTP SENT [PASV] + FTP RCVD [227 Entering Passive Mode (128,59,48,24,246,37)] + FTP SENT [RETR BIGFILE] + FTP RCVD [550 BIGFILE: Value too large for defined data type.] + +Created the same 3GB on a Tru64 Unix system that allows FTP access. Made +the connection from C-Kermit on Solaris (32-bit with LFS): + + 16:46:12.908 FTP SENT [SIZE BIGFILE] + 16:46:12.947 FTP RCVD [213 3000000001] + +Note that it takes less than half a second to get the reply. Now I start +the download and then interrupt it at about 2%: + + 16:46:12.979 FTP SENT [TYPE I] + 16:46:13.174 FTP RCVD [200 Type set to I.] + 16:46:13.226 FTP SENT [PASV] + 16:46:13.262 FTP RCVD [227 Entering Passive Mode (15,170,178,171,11,37)] + 16:46:13.299 FTP SENT [RETR BIGFILE] + 16:46:13.337 FTP RCVD [150 Opening BINARY mode data connection for BIGFILE..] + 16:47:24.895 FTP RCVD [426 Transfer aborted. Data connection closed.] + 16:47:24.934 FTP RCVD [226 Abort successful] + 16:47:24.991 FTP SENT [MDTM BIGFILE] + 16:47:25.028 FTP RCVD [213 20060706204458] + +Now I do a REGET: + + 16:51:55.321 FTP SENT [PASV] + 16:51:55.357 FTP RCVD [227 Entering Passive Mode (15,170,178,171,11,43)] + 16:51:55.394 FTP SENT [REST 122736640] + 16:51:55.430 FTP RCVD [350 Restarting at 122736640. Send STORE or RETRIEVE..] + 16:51:55.431 FTP SENT [RETR BIGFILE] + 16:51:55.469 FTP RCVD [150 Opening BINARY mode data connection for BIGFILE..] + +This worked perfectly, as far as I can tell; the FT display picked up in the +right place; the thermometer, percent done, and estimated time remaining +were the same as when we left off last time. I did the same thing several +more times, everything was OK. It would have taken a really long time to +let this run to completion, but I think this demonstrates that Scott's +symptoms are server-dependent. No changes. 6 July 2006. + +Checked current code on VMS 8.2-1 on IA64 / UCX 5.5, builds fine. +No changes. Updated listing at HP. 6 July 2006. + +Checked FTP GET of large file in ASCII mode against Tru64 FTP server. It +was fine, and there was no delay in the server's response to our SIZE command +(as there would be if it were scanning the entire file to count how many +bytes would be required to send it in text mode). 7 Jul 2006. + +Tested FTP PUT big file against Tru64, OK. Ditto FTP RESEND big file: + + C-Kermit>resend BIGFILE + PUT BIGFILE (binary) (3000000001 bytes)---> PASV + 227 Entering Passive Mode (15,170,178,171,13,186) + ---> SIZE BIGFILE + 213 343211280 + ---> MDTM BIGFILE + 213 20060707141243 + ---> APPE BIGFILE + 150 Opening BINARY mode data connection for BIGFILE (128.59.59.56,45470). + +Made REPUT a synonym for RESEND. ckuusr.c, 7 Jul 2006. + +Added FTP REPUT and FTP RESEND since previously there was no FTP-prefixed +command for recovering uploads, only the regular RESEND command, which might +not have been obvious to people. ckcftp.c, 7 Jul 2006. + +Added help text for FTP RESEND and REPUT and amended RESEND help text. +ckcftp.c, ckuus2.c, 7 Jul 2006. + +Changed name of \fpicture() to \fpictureinfo() and added help text. By the +way, ImageMagick can do this too: identify -format "%w %h" dscf0520.jpg. +The advantage of having it in Kermit is that not everybody has ImageMagick. +ckuus[24].c, 7 Jul 2006. + +Changed the numeric comparisons = < > <= >= != to allow long integers by +changing the data type to CK_OFF_T, etc. ckuus6.c, 7 Jul 2006. + +Noticed that \fkeywordvalue(foo=this is a string) only kept the first word. +Fixed it to keep the whole definition. Also added \fkwvalue() as a briefer +synonym. ckuus4.c, 7 Jul 2006 + +Sometimes we want to check if a file's status before we've FOPEN'd it, in +which case the channel variable is likely to be empty and \f_status(\%c) +would get an error. Making the obvious change didn't fix this, however. It +turns out that the function evaluator failed to adjust argn (argument count) +when trailing arguments were empty, and argn was being used in this case, +and probably others, to test whether an argument existed. I added code to +adjust argn to reflect the number of arguments up to and including the +rightmost non-empty one. ckuus4.c, 7 Jul 2006. + +Fixed \fstripb() to not dump core if second argument is missing. +ckuus4.c, 7 Jul 2006. + +Discovered that it was not obvious what pattern to use to match strings +enclosed in square brackets. "if match [abc] \[*\]" didn't work. Neither +did various other tricks like NCRs for the brackets. However, "if match +[abc] \\[*\\]" does work. Trying to fix this would no doubt break 100 other +things, so let's call it a feature. 7 Jul 2006. + +Added \fgetpidinfo(n) to return info about a process ID; for now it simply +returns 1 if the process is alive and 0 if not (or -1 if the argument is +bad or missing or on any kind of error). ckuusr.h, ckuus[24].c, 7 Jul 2006. + +The "where-did-my-file-go" message seemed to be ending with a LF rather +than CRLF, probably because the terminal modes had not yet been restored, +leaving the next prompt hanging below it, rather than on the left margin, +if C-Kermit exited immediately after the transfer. Fixed by changing +all \n's to \r\n's in wheremsg(): ckcpro.w, 8 Jul 2006. + +Added \v(lastkwval) so we can retrieve programmatically the keyword most +recently processed by \fkeywordval(). ckuusr.h, ckuus4.c, 9 Jul 2006. + +--- Dev.20 --- + +Added #ifdef SV68, #include , #endif because Unix System V/68 on +Motorola choked on the SEEK_CUR reference without it. ckuus4.c, 10 Jul 2006. + +Make \fkeywordval(xxx) undefine xxx (i.e. when a keyword is given with no +value). This way command-line keywords will always override preexisting +default definitions, whether they have a value or not, which makes it easier +to parse command lines like "foo=bar blah xx=yy". ckuus[24].c, 12 Jul 2006. + +On 29 Nov 2005 I changed IF KERBANG to solve a problem (see entry for that +date), but introduced a new one; namely that you can't have (e.g.) a FATAL +macro that uses IF KERBANG to decide whether to EXIT all the way or STOP +back to the prompt. Changed it again, this time to require not that the +command level be 1, but that the command *file* level be 0 (i.e. that we are +in the top-level command file, irrespective of the command or macro level, +but not in a subfile). ckuus6.c, 12 Jul 2006. + +It is unhelpful when Kermit gets a syntax error in the middle of a big +compound statement block (e.g. FOR or WHILE loop) and dumps out the whole +thing in an error message. I changed the two places where this can happen +to call a new routine that, instead of dumping out the entire cmdbuf, +checks its length first and if it's more than a line long, truncates it +and adds an ellipsis. ckuus6.c, 12 Jul 2006. + +The new RENAME command didn't give very good error messages, e.g. if the +filespec didn't match any files. Fixed in dorenam(): ckuus6.c, 12 Jul 2006. + +Fixed DIR /TOP to work if the /TOP:n argument was omitted, defaulting +to 10. domydir(): ckuus6.c, 12 Jul 2006. + +Added DIR /COUNT:v to count the number of files that match the given +criteria and store result in the variable v. ckuusr.h, ckuus[r26].c, +24 Aug 2006. + +Added HDIRECTORY as an invisible synonym for DIR /SORT:SIZE /REVERSE. +Can be used with other switches, of course, so (e.g.) HD /TOP shows the +ten biggest files. ckuusr.h, ckuus[r26].c, 24 Aug 2006. + +DIR /FOLLOWLINKS and /NOFOLLOWLINKS always did the same thing; the switch +was ignored, a symlink is always followed. Fixed in ckuus6.c, 24 Aug 2006. + +Added DIR /NOLINKS, which means don't show or count symlinks at all. +ckuusr.h, ckuus[r26].c, 24 Aug 2006. + +Build on Solaris 9 and NetBSD 3.0, 24 Aug 2006. + +Added a missing definition for LOCK_DIR in the Linux HAVE_BAUDBOY case, +suggested by Gerry Belanger. ckutio.c, 6 Oct 2006. + +Suggested by Jim Crapuchettes: \v(dialmessage) is the text string +corresponding to \v(dialstatus). ckuusr.h, ckuus4.c, 6 Oct 2006. + +Soewono Effendi sent code for exit sequence to leave DTR on; this amounted +to unsetting HPUCL in c_cflag. I did it a simpler way, hopefully portable +to all Unixes, but who knows at this late date. The code is inside +#ifndef CK_NOHUPCL..#endif in case it causes trouble. It is executed if +SET EXIT HANGUP is OFF and a serial port was open at the time Kermit exits +(or closes it explicitly). ttclos(): ckutio.c, 6 Oct 2006. + +Built on Solaris9/Sparc; FreeBSD 6.2/AMD64; NetBSD 3.0/i386; HP-UX 11i v2; +SCO OSR6.00. + +--- Dev.21 --- + +Added netbsd+openssl target to makefile. Built OK (NetBSD 3.0, OpenSSL +0.9.7d) except with some warnings in ck_crp.c. Connects and logs in OK to a +secure site. 10 Oct 2006. + +Added a debug statement to ftp_hookup() to record the TCP port that was used. +ckcftp.c, 11 Oct 2006. + +Built with OpenSSL 0.9.7l on Solaris 9. Built with OpenSSL 0.9.8d on +Solaris 9; connects and logs in to a secure site. 11 Oct 2006. + +The new RENAME command didn't work if both the source and destination names +included directory segments, e.g. "rename /tmp/foo ~/bar" (see notes of +4 May 2006). This was fixed in renameone() by a special case in which +the second argument is given but it is a filename, not a directory name. +ckuus6.c, 11 Oct 2006. + +Fixed unguarded reference to dialmsg[] for \fdialmessage(), noticed by +Gerry Belanger. ckuus4.c, 12 Oct 2006. + +Added a TOUCH command that does what UNIX touch does: creates the file if it +does not exist, updates the timestamp if it does. If a wildcard is given, +it operates only on existing files. It shares the DIRECTORY command parser, +so all the same file selection switches can be given. ckuusr.[ch], +ckuus[26].c, 12 Oct 2006. + +PeterE noticed that if you FOPEN a file, do some seeks or reads, then FCLOSE +it, then FOPEN it again (or open a different one), some of the old +information is still there (e.g. current line number). This is an artifact +of the changes of May 4th. Now the file closing and opening routines are a +bit more careful about scrubbing and initializing the file info struct. +ckuus7.c, 12 Oct 2006. + +--- Dev.22 --- + +Built OK on Red Hat Linux AS4 with both "make linux" and "make linuxnc". +15 Oct 2006. + +DIRECTORY /BRIEF ignored file selection switches and always listed all +files. This was because of how I cleverly called filhelp() (the routine +that lists matching files when ? is typed in a filename field) and, of +course, filhelp() doesn't know anything about the DIRECTORY command's file +selections. Changed filhelp() to accept all the args needed for passing +along to fileselect(), renamed it to xfilhelp(), and made a filhelp() stub +that chains to xfilhelp() with null selections. ckcker.h, ckucmd.[ch], +ckuus6.c, 29 Nov 2006. + +SHOW CONNECTION for an SSH connection said the connection type was "NET" +rather than "SSH". Fixed in dolognet(): ckuus3.c, 29 Nov 2006. + +SHOW CONNECTION didn't show the TCP port number. This command works by +parsing the current connection log entry string, which doesn't have a field +for this, but which sometimes shows the port number as part of the hostname +(but more often not). Added code to dolognet() to log the TCP port number, +if known. This involved adding a gettcpport() function to ckcnet.c. +ckcnet.[ch], ckuus3.c, 29 Nov 2006. + +This was impossible: def \%1 upper, echo \f\%1(abc) -- i.e. to "compose" a +function name. Fixed in zzstring(). But note that it's still not possible +to do this: def \%1 \fupper, echo \%1(abc) -- because at the point where +"\fupper" is encountered, which is automatically fed to fneval(), the +argument list hasn't been read yet. ckuus4.c, 29 Nov 2006. + +The meaningless Lisp command (=) would cause Kermit to hang. Due to some +idiosyncracy in the parser, it would see this as ((=) and would go into +"wait for the closing paren" mode. There was already a hack in the code to +compensate for this, but it didn't work. I fixed the hack but I don't +understand the real problem. Anyway, comparing Kermit with real (Franz) +Lisp I discovered that comparison operators do not require two arguments, as +Kermit has been doing, although they do require at least one. I changed +Kermit to not require two, so now all the comparison predicates behave +exactly like Franz Lisp, including getting an error if there are no args). +ckuus[r3].c, 29 Nov 2006. + +From to-do list: Make a way to inhibit pattern matching in SWITCH labels. +It's already there; just quote the wildcard characters; the only trick is +that for some reason (such as that SWITCH is really an internally defined +macro), a double quote is needed: + + switch \%1 { + :a\\*z, echo literally "a*z", break + :abcxyz, echo literally "abcxyz", break + :a*z, echo a...z, break + :default, echo NO MATCH + } + +In first case, the asterisk is taken literally; in the third it's a +metacharacter and the label matches any string that starts with 'a' and +ends with 'z'. + +Array initialization would quit early if any initializers were undefined, +e.g. "decl \&a[] = \%a \%b \%c" would stop at the first element if \%b +was not defined, even though \%c might be defined. Fixed in dodcl(): +ckuusr.c, 30 Nov 2006. + +DIR /ARRAY:a filespec, when the filespec does not match any files, +terminates with the array undeclared. It would be better to return a +declared but empty array (\&a[0] = 0). The code is already there to do +that, but isn't working. And yet "declare \&a[0]" does indeed create a +0-element array ("show array" shows a dimension of 0). Turns out there were +two problems; one was the careless recycling of a local variable ("array"), +resulting in failure to create \&a[] (but not any other array). Fixed in +domydir(): ckuus6.c, 30 Nov 2006. + +The other problem was that dclarray(), when called with an array name and a +dimension of zero, does two different things depending on whether the array +already existed. There is still a fair amount of confusion about whether a +dimension of 0 indicates an array with 1 element (as it should) or a +nonexistent array. We call dclarray() with a size of 0 to undeclare an +array but we also need to able able to declare an array with only element 0. +I changed dclarray() to treat a negative dimension as a command to destroy +the array, and 0 or positive as a command to create the array with the given +dimension. ckuus[r56].c, 30 Nov 2006. + +Next problem: when chkarray() returns 0, this should not be interpreted to +mean the array does not exist. Looks like the only place this happened was +in \fcontents(); fixed in ckuus4.c, 30 Nov 2006. + +If we include file selectors with DIR /ARRAY:&a and some of the files that +match the given filespec but don't fit the selectors, the array's dimension +is bigger than its number of elements. Added code at the end of domydir() +to resize the array so \fdim() returns the number of filenames in the array, +and also made sure that element 0 contains that number too. ckuus6.c, +30 Nov 2006. + +This would be a nice elegant way to loop over a bunch of files, if it worked: + + for \%i 1 \ffiles(*) 1 { rename \fnextfile() xxx_\flpad(\%i,3,0) } + +But in this loop, Kermit skips every other file (beginning with the first) +and then runs out of files halfway through the loop. Why? Because in +commands like RENAME and DELETE, the filename parser is in a chained FDB +with the switch parser. First the switch parser, cmswi(), gets its hands on +\fnextfile(), passing it through the evaluator and thus getting the first +filename, which it then sees is not a switch, so now the field is parsed by +the next parser in the chain, cmifi(), which causes \fnextfile() to be +executed again. In fact, the FOR loop has nothing to do with; the same +thing happens like this: + + void \ffiles(*) + delete \fnextfile() + +This deletes not the first file, but the second one. Obviously users can be +told not to refer to \fnextfile() in chained-fdb fields: + + for \%i 1 \ffiles(*) 1 { .f := \fnextfile(), delete \m(f) } + +but this is hardly intuitive. I had some clever ideas of how to make +\fnextfile() work as expected in this context but it's way too much magic. +Better to simply document that \fnextfile() is "deprecated" and the array +format should be used: + + for \%i 1 \ffiles(*,&a) 1 { delete \&a[\%i] } + +The difference is, an array element doesn't change every time it's referred to! + +Added a /PRESERVE switch to the COPY command to preserve the timestamp and +permissions of the file. I did this using the Kermit APIs so it should work +for any version of C-Kermit or K95. ckuus[26].c, 30 Nov 2006. + +Added COPY /OVERWRITE:{ALWAYS,NEVER,OLDER,NEWER} to control name collisions +when copying across directories. ckuus[26].c, 1 Dec 2006. + +--- Dev.23 --- + +Fixed a bug in SET TELNET PROMPT-FOR-USERID, SET AUTH KERBEROS[45] PROMPT, +and SET AUTH SRP PROMPT in which the user's string was compared with a +literal (s == ""), reported by Pavol Rusnak. Worse, empty strings (if the +test succeeded) were turned into null pointers, and then fed to strlen(). +Fixed in ckuus3.c, 5 Dec 2006. + +Added an optional 4th argument to \findex(), \frindex(), \fsearch(), and +\frsearch(): the desired occurrence number of the searched-for string. +\frsearch() was a bit tricky. ckuus[24].c, 7 Dec 2006. + +Added \fcount(s1,s2) to tell the number of occurrences of s1 in s2. +ckuus[24].c, 8 Dec 2006. + +Added \ffunction(s1) to tell if a given built-in function is available. +ckuus[24].c, 8 Dec 2006. + +Changed RENAME /COLLISION:PROCEED to be /COLLISION:SKIP, which is clearer. +ckuus[26].c, 8 Dec 2006. + +For communication protocols: INPUT /COUNT:n to read exactly n characters +without any matching. Can be used, for example, with CONTENT_LENGTH in CGI +scripts; NUL characters are counted but not collected. ckuusr.[ch], +ckuus4.c, 8 Dec 2006. + +There was a bad bug in the date-parsing routines; it's been there for years. +If a date string includes a timezone, e.g. "Sat, 9 Dec 2006 19:26:23 EST", +and converting to GMT changes the date, the variables for day, month, and +year (which are used later) were not updated, and the final result was a day +off. Fixed in cmcvtdate(): ckucmd.c, 10 Dec 2006. + +Built OK with SSL/TLS. Tested with the POP script, found that I broke INPUT +when adding the /COUNT feature; there was a path through the code that could +leave the "anychar" variable unset and therefore random. Fixed in +doinput(). The POP script, which does not use /COUNT, works again and so +does a new CGI script, which does use /COUNT. ckuus4.c, 10 Dec 2006. + +Supplied a missing comma in the help-text array for HELP SET TERMINAL, which +resulted in bad formatting in K95 around SET SNI-FIRMWARE-VERSIONS. +ckuus2.c, 10 Dec 2006. + +Made "help locus" a synonym for "help set locus". ckuusr.[ch], ckuus2.c, +11 Dec 2006. + +This morning the Columbia FTP server was malfunctioning in a perfect way +for me to implement and test an FTP timeout mechanism. The server would +close the data connection after sending the file, but the client never saw +the close and was stuck forever in a recv(). I added code to do a select() +on the data connection prior to entering the recv(), with a timeout on the +select() that the user can establish with SET FTP TIMEOUT. Built and tested +on Solaris 9, clear-text FTP. Also built cleanly for FTPS and tested +against a server that does not hang; I don't have access to an FTPS server +that would tickle the timeout code. ckcftp.c, 11 Dec 2006. + +--- Dev.24 --- + +Fixed a bug in the INPUT /COUNT: parser: the array of search strings was +never initialized, which didn't matter before, but with /COUNT:, if the +first element was not a NULL pointer, we'd treat it as a search string, and +then if it happened to match something in the input stream, the operation +would stop before the count was exhausted. Fixed by (a) initializing the +array, and (b) ignoring any search strings if /COUNT: was given. ckuusr.c, +13 Dec 2006. + +Removed a debug() statement from zsattr() that suddenly started making some +version of gcc complain, reported by Gerry Belanger. ckufio.c, 13 Dec 2006. + +--- Dev.25 --- + +Some casts for the 3 interior args of the new select() call in ckcftp.c +for HP-UX 9. 14 Dec 2006. + +Changed \fkeywordvalue() to accept a string rather than a single word +as its second argument, so that more than one separator could be specified, +and to return -1 on error, 0 if it found nothing, 1 if given a keyword but +no value, and 2 if there was a keyword and a value. dokwval(): ckuus[24].c, +14 Dec 2006. + +Checked FTP timeout on command channel with FTP DIRECTORY of a big directory +using a path into our ftp server that preserves the hanging behavior. The +timeout was actually working, but the failure condition wasn't propagating +back to the user, and there was no error message. Fixed in doftprecv2() and +failftprecv2(): ckcftp.c, 15 Dec 2006. + +Added the obvious timeout checks for FTP uploads, but I have no way to test +the code since our misbehaving FTP server does not hang when receiving +files, only when sending them. But uploads work both with and without a +timeout set, so at least no harm is done. ckcftp.c, 17 Dec 2006. + +When downloading with FILE DESTINATION NOWHERE (= /CALIBRATE), Kermit still +checked the size of the incoming file and refused it if there wasn't enough +free disk space, on platforms (such as VMS) where zchkspa()) actually works; +reported by Bob Gezelter. ckcfn3.c, 18 Dec 2006. + +Built on Mac OS X 10.4.8 and NetBSD 3.1_RC3, all OK. 19 Dec 2006. + +--- Dev.26 --- + +Built on VMS 7.3-2/Alpha. Had to squelch a couple compiler warnings by +changing some ints in the new \fpictureinfo() code from unsigned to signed, +and fix a typo in the prototype for the new gettcpport() function. +ckcnet.h, ckuus4.c, 22 Dec 2006. + +--- Dev.27 --- + +Parameterized pty routines and all references to them for file descriptor, +rather than to use global ttyfd, thus allowing ptys to be created for +different purposes. Tested on Solaris 9 and Mac OS X 10.4.8, with "set host +/connect /pty emacs" (fine in both cases), and (more to the point) "set host +/connect /pty kermit" -- here we make a connection from one Kermit process +to another and transfer a file; works fine and wasn't especially slow either; +a good sign. ckcdeb.h, ckutio.c, ckupty.c, 22 Dec 2006. + +Created a new version of ttruncmd() called ttyptycmd(), which works by +calling do_pty() to get a pty to run the command on, and then in a loop, +reads from the pty and writes to the net and reads from the net and writes +to the pty, using select() to which of those it should do on each pass. +First cut just uses single-byte reads and writes. Tested using Kermit +itself as an external protocol. Works but slowly: 6000cps. Zmodem doesn't +work at all. ckutio.c, 24 Dec 2006. + +Changed single-character read() and write() to buffered reads and writes, +with ttxin() and ttol() used for network i/o. Using Kermit as the external +protocol, this gives 450Kcps (about 1/3 normal on this connection). + +But now there's a problem: the loop doesn't know when to stop. How does it +know when the process that is running on the pty has exited? With single +character read()'s that are executed unconditionally when select() says the +pty has data waiting, as in the first pass, I get EIO if there actually +isn't any, and can exit the loop. But now, to avoid blocking, I call +in_chk() to see how much data is waiting, and I don't try to read anything +if it says nothing is waiting. If the process associated with the pty file +descriptor has terminated, in_chk() would presumably get some kind of error, +but it doesn't. I changed do_pty to return the pid of the fork where it +execs its command so we can check the pid with kill(pid,0) when in_chk() of +the pty says 0, but this doesn't help either; it seems like the process is +not exiting, but of course it is. + +I could not find any legitimate way to test when the pty fork terminated. +Select() always says the pty file descriptor was ready, no matter what. +Select() never reports an exception on the pty file descriptor; +in_chk(ptyfd) returns 0 and not an error. read(ptyfd,...) gets 0 but not an +error. fcntl(ptyfd,...) doesn't get an error. Finally I tried +write(ptyfd,c,0) and this indeed gets EIO (i/o error). With this, using +Kermit as the external protocol works fine in Solaris but I tend to think +this trick will not be very portable (it isn't). 24 Dec 2006. + +Made ttptycmd() use a more intelligent buffering scheme, fixed a few things +about how I was setting up the select() call that should address some of +yesterday's problems. Still doesn't work but it's progress. A: 25 Dec 2006. + +Debugging yesterday's code... Still, the error conditions are never set, +we never detect when the pty closes. In Solaris, if select() says ptyfd is +ready to read but in_chk() says there are no characters there, we can treat +this as a loop-exit condition. But in NetBSD, in_chk() always says 0 when +used on a pty (but works OK on a serial or net connection). + +Realized I could not use in_chk() on the pty because there is too much +baggage with the communication path -- myread(), etc etc) -- so I replaced +this with a simple ioctl(ptyfd,FIONREAD,&n). This works fine in Solaris but +always returns 0 in NetBSD, despite what the man page says (i.e. that this +function can be used on any file descriptor). + +OK, let's see.... select() does not return useful results. It says +characters are waiting on ptyfd when they are not, and it never detects the +closure of the pty..... Well of course not, because we are the ones who +have to close it. Just because the process has stopped doesn't mean the pty +is closed. So we're back to square one, how do we know when to close it? +ckupty.c seems to keep the process ID in a global variable, pty_fork_pid +(which is not the same as the pid now returned by do_pty(), which is +useless, but I don't understand why). But it doesn't matter because when we +kill(pty_fork_pid,0), we still get no error of any kind, even after we know +the process has exited. I am completely flummoxed. select() lies, and even +if it didn't, there is simply no completion criterion. In the loop, +select() always says that the pty is ready to read. To be continued. +26 Dec 2006. + +Back to Square One, single-byte reads and writes. + + . This works for both ripple and Kermit. + . Doesn't work for Zmodem but we'll deal with that later. + . In this case FD_ISSET(ptyfd) is still true after pty closes. + +But the ensuing read() gets EIO so we know the pty is gone. That means the +same thing should happen in the buffered version, no? Yes; I went back to +the buffered version and replaced all the other nonworking tests by a +blocking read of 1 byte on the pty and this detects the termination. Now: + + . ripple works perfectly (of course it's only one-way). + . Kermit fails + +Let's call the remote, forked, redirected, external Kermit A and its +local partner B. A sends its S-packet, B receives it OK and Acks. +A apparently does not receive the ACK in time, so sends the S again, but OK. +followed immediately by the F. B Acks the F. A sends the A, B Acks it. +But now A sends a piece of the previous F packet and the the first piece +of a D packet. + +Clearly the buffering is messed up. Sure enough, there was an extraneous +statement incrementing a read pointer in a write section. Removing that +cleared up the problems with Kermit, now we can send and receive substantial +files efficiently in remote mode. Zmodem seems to work too, except that at +the beginning a bunch of "**B0800000000022d"'s are stuffed into Kermit's +command buffer, so after the transfer we get some error messages. + +In local mode, over a Telnet connection, Kermit works fine. Zmodem works +OK too except it doesn't finish right, so at the very end rz on the far end +is still waiting for something; if I cancel out of it with ^X^X^X^X^X, it +deletes the file. So there still is something wrong with the termination +test. + +Also you don't see anything on your screen when running Kermit or Zmodem +this way. That's to be expected, since they are using stdio for the +transfer, so they can't also be displaying progress or other messages. + +Built this on NetBSD again... Seems to work this time, but has trouble +finishing, like Zmodem. Hmmm, on closer examination, it turns out that +since in_chk() always returns 0 on the ptyfd, we fall into our new +single-byte read code, so it's really slow, like 10K cps on a connection +where 1M is the norm. 27 Dec 2006. + +Switched the pty from buffer peeking (FIONREAD) and blocking reads to to +nonblocking reads (O_NONBLOCK / O_NDELAY). Works just fine on NetBSD except +now we no longer get EIO at the end when trying to read from the pty process +that has exited. In fact, we're back to square one again. not ioctl(), not +fcntl(), not select(), not even read() gets an i/o error after the pty +process exits. But in NetBSD, we have to use nonblocking reads because ... +Hmmmm, maybe switch the fd between blocking and nonblocking for the test... +Nope, NetBSD seems to be hopeless (later, Ed Ravin confirmed that similar +problems have been observed with other applications that try to do this). + +Switching to Linux, I see that yesterday's Solaris code (blocking reads) +works exactly the same way on Linux. + +Tried today's O_NDELAY method on Solaris. It works perfectly. And then I +moved this one to Linux and it works perfectly there too. Except in both +cases we have the weird thing with Zmodem at the end, but I think that's +because rz/sz don't use standard i/o. On NetBSD, it still hangs at the end. + +Turns out that testing the pid works in NetBSD, even though it didn't in +Solaris. Turns out read() gets an i/o error in Solaris and Linux but not +in NetBSD. So checking the read result first, and then checking the pid +if read() got zero bytes catches all three. 28 Dec 2006. + +Now the question of return code. In the original ttruncmd() function, we do +a fork() and a wait(). When the external protocol program finishes, wait() +gives us its return code and we can pass it on through \v(pexitstat) as well +ttruncmd's own return code. But ttptycmd() has to interact with the pty +continuously, so it can't just sit back and wait() for it. Instead we have +to detect when the process has exited and then call waitpid() on the fork +pid, before shutting down the pty. Tested on Solaris using Kermit as the +external protocol and then inducing failure, or letting it run to +completion. FAILURE and SUCCESS set appropriately in each case. Tested +with Zmodem too, works OK except for the aforementioned cosmetic glitch at +the end. Tested on NetBSD, all OK. + +To make K5 connection to Panix from Spam: + + set telnet debug on + authenticate K5 init /realm:PANIX.COM /password:xxxxx + set host shell.panix.com 23 /k5login + +Good... Now I try to send a file from Spam to Panix over the K5 connection +using Kermit itself as the external protocol. It fails. Inspection of the +debug log on the far side shows that the S-Packet was received correctly, +good! This means we are reading the clear-text S-Packet from the external +Kermit program, and that ttol() is encrypting appropriately. + +The remote Kermit sends the Ack and goes to read the next packet: ttinl() +calls myfillbuf() and: + + SVORPOSIX myfillbuf calling read() + SVORPOSIX myfillbuf=0 <-- read returns 0 + SVORPOSIX myfillbuf ttcarr=2 + SVORPOSIX myfillbuf errno=0 <-- and reports no error + HEXDUMP: mygetbuf read (-3 bytes) + mygetbuf errno=0 + ttinl myread failure, n=-3 + ttinl myread errno=0 + ttinl non-EINTR -3[closing] + +This happens because myfillbuf() deliberately returns -3 when read() gets 0 +bytes. I don't understand why this happens but the real problem is yet to +come. The local Kermit (the one that has made the secure connection and is +running the external protocol through ttptycmd()) eventually figures out +that the transfer failed and when we reconnect, we get total garbage -- the +encryption either stopped happening, or got out of sync. + +Looking at the local debug log, ttol() is doing its job, converting the +initial "kermit -r\13" from plaintext to cyphertext, as shown by the +hexdumps. Then it enters ttptycmd()... Hmmmm, wait, how can it send the +"kermit -r" before it starts the external protocol? Never mind, worry about +that later... Anyway, ttptycmd() says: + + ttptycmd loop top have_pty=1 + ttptycmd loop top have_net=1 + ttptycmd FD_SET ptyfd in + ttptycmd FD_SET ttyfd in + ttptycmd nfds=5 + ttptycmd select=1 + ttptycmd FD_ISSET ttyfd in + ... + ttptycmd in_chk(ttyfd) n=11 + ttptycmd ttxin n=11 + +ttxin() asks for 11 bytes, myfillbuf() gets 11 bytes, and hexdump() shows +the cyphertext, there doesn't seem to be any decrypting going on. Hmmm, it +looks like the regular code calls ttinc() in a loop, rather than ttxin(). +Maybe ttxin() doesn't have decryption hooks. No, that's not it, the code is +there, but the Kermit packet reader does not use ttxin(), it uses ttinl(). +But of course we can't use that for external protocols because it's designed +only to read Kermit packets. Substituting a loop of ttinc()s for the ttxin() +call fixes things (and strangely enough, it seems to be faster). And now we +have our first external protocol transfer over a secure connection (external +Kermit program, Linux over Kerberos 5 to NetBSD). Zmodem worked too for a +short file but "something happens" with longer ones. 29 Dec 2006. + +New makefile target for Linux with Kerberos 5, linux+krb5, that doesn't +include anything extra from SSL or other security methods (but apparently it +is still necessary to include -DOPENSSL_097 in order to get the right names +for the DES routines?). Ditto netbsd+krb5 for NetBSD, except in this case +-DOPENSSL_097 is not necessary. makefile, 30 Dec 2006. + +Note to myself: On Panix: + + export LD_LIBRARY_PATH=/usr/local/kerblib + make netbsd+krb5 "K5LIB=-L/usr/local/kerblib" "K5INC=-I/usr/local/include" + +Can't telnet-k5 from newly built Kermit on NetBSD; partway through the +negotiations, just after "TELNET RCVD SB ENCRYPTION SUPPORT DES_CFB64 +DES_OFB64 IAC SE" it dumps core. The last two lines in debug.log after +this are: + + tn_sb[len]=5 + encrypt_support[cnt]=2 + +Rebuilding with -DOPENSSL_097 doesn't change anything. Ed Ravin said they +have two different Kerberos installations, Heimdal and MIT; maybe some +mixup between the two explains the problem (Jeff concurs). The core dump +occurs in ck_crp: encrypt_support(): + + debug(F100,"XXX ep not NULL","",0); + type = ep->start ? (*ep->start)(DIR_ENCRYPT, 0) : 0; <-- Here + debug(F101,"XXX new type","",type); + +Anyway, I can log in with Kerberos 5 to Panix OK from Columbia (sesame) +using 8.0.201. So let's try to resurrect the Solaris version with everything: + + solaris9g+krb5+krb4+openssl+shadow+pam+zlib + +I hunted around to find where the current library and header file +directories were... Last time I tried this (March 2006) it bombed, not +finding libdes. Instead we have /opt/kerberos5125/lib/libdes425.a. Made a +new cu-specific target that includes this; now we get farther; it blows up +in ckcftp.c with tons of errors and warnings, which we can worry about +later. Building again with -DNOFTP, it gets to ckuath.c (the first security +module) and: + + ckuath.c:151:18: error: krb5.h: No such file or directory + ckuath.c:152:21: error: profile.h: No such file or directory + ckuath.c:153:21: error: com_err.h: No such file or directory + ckuath.c:176:28: error: kerberosIV/krb.h: No such file or directory + In file included from /opt/openssl-0.9.8d/include/openssl/des.h:101, + from ckuath.c:219: + +Found krb5.h in /opt/kerberos5125/include/krb5.h, added a -I for this +directory ... Now we get lots of warnings in ckuath.c, but it completes OK, +then we wind up bombing out in ck_crp.c; I don't know why -- there are all +the same warnings (related to argument passing to DES functions), but no +errors. I have no clue. + +Tried to resurrect the solaris2x+krb4 target; this required changing -lkrb +to -lkrb4 and -ldes to -ldes425. Lots of warnings in ckutio.c, ckcnet.c, +ckctel.c, then it bombs out in ckcftp.c because it can't find krb.h. I +found it, adjusted the -I flags, but now it bombs because krb.h itself +#includes , which of course it can't find because the +brackets mean it's looking in /usr/include/kerberosIV/, which, of course, +the sys folks have removed. Giving up on Solaris again. Later, Jeff said +"Solaris does not publicly export the krb5 libraries. You need to build +the MIT Kerberos libraries separately and link to them." 30 December 2006. + +Changed copyright date to 2007. ckcmai.c, 1 Jan 2007. + +With Ed Ravin's help, successfully built C-Kermit with Kerberos 5 and +OpenSSL (netbsd+krb5+openssl+zlib), but it does not make K5 connections; it +gets hung up in the Telnet negotiations. 3 Jan 2007. + +Downloaded MIT Kerberos 5 v1.4.4 to Solaris 9, 54MB worth. This is just so +I can build a Kerberized C-Kermit for testing ttyptycmd(). Ran the +configure program, got a few warnings but it didn't fail (should it?) Did +"make install", specifying a private directory but it failed immediately +with "cannot stat libkrb5support.so.0.0: No such file or directory". +OK, I tried. 3 Jan 2007. + +Made a new makefile target for Mac OS X, macosx10.4+krb5+ssl, ran it on Mac +OS X 10.4.8. It bombs out in ckcftp.c with: ckcftp.c:551: error: static +declaration of 'gss_mech_krb5' follows non-static declaration +/usr/include/gssapi/gssapi_krb5.h:76: error: previous declaration of +'gss_mech_krb5' was here". Ditto for gss_mech_krb5_old, gss_nt_krb5_name, +and gss_nt_krb5_principal. Tried again with -DNOFTP. We get lots of +warnings in the network modules, but they complete. But ck_ssl.c bombed +with a conflict between its own declarations of encrypt_output and +decrypt_input and the ones in ckuat2.h; removed the prototypes from the +latter (as Jeff advised) it built OK and it works OK too. Built with FTP +too, but with link-time warnings about the aforementioned gss_* symbols. +#ifdef'd them out (gss_mech_krb5, gss_mech_krb5_old, gss_mech_name, and +gss_mech_principal) for MACOSX, where these symbols are exported by the +library. Now it all compiles and links OK, and runs OK too. 3 Jan 2007. + +Spent a day hunting around for a version of Zmodem that would build and +execute on Mac OS X, finally found one. Now at last I could try a Zmodem +external-protocol transfer over a secure connection. But phooey, C-Kermit's +pty support didn't work on this box. Kermit finds master /dev/ptypa OK, +then in ptyint_void_association() tries to open /dev/tty but gets ERRNO=6 +"device not configured" (which is apparently OK, because the same thing +happens on other platforms where this works), then tries to open slave +/dev/ttypa and gets ERRNO=13 "permission denied" because, indeed, I don't +have r/w permission on the device. Left a message. 4 Jan 2007. + +Changed TRANSMIT /BINARY output buffer size from 252 to 508 to avoid +TCP fragmentation. Need to add a SET command for this later. +ckuus4.c, 5 Jan 2007. + +Found another Mac where the ptys weren't protected against me, make a K5 +connection and transferred a largish file with Zmodem with zero glitches, +except it was kind of slow, 84K cps. Well, we're doing single-character +reads on the net (ttinc()'s instead of ttxin()). Hmmm, but then I did it +again and got 2.2Mcps. Success was reported, but it actually didn't work; +it only sent the first quarter of the file.... Oh well, at least now we +have a testbed. 5 Jan 2007. + +Tried again, saw that the file is actually transferred instantly but then +we're not picking up the protocol at the end. Theory: after the transfer +finishes, we come back to the prompt on the remote host, which means we have +something to read from the net and write to the pty, but the pty has already +exited. AFTER THE PTY IS GONE, WE DO NOT WANT TO READ FROM THE NET ANY +MORE. Adding this test makes Kermit succeed right away when sending the +same largish file, with a transfer rate of 4M cps, that's better. But the +rz program on the far end is evidently not receiving the goodbye handshake +from the receiver, because it sits there foreever in its *B09002402009418 +mode until I ^X^X^X^X^X out of it, at which point it deletes the file it +already received, not very helpful. In the code, I read from the pty if the +pty is open and there is room in the buffer. This means that when we get to +the end, either there is no room in the buffer (unlikely) or the last bit +sent by sz before exiting was cut off when the fork closed. Why do we get +in this fix only with Zmodem and not with Kermit? + +In Mac OS X, after sz exits, we get ERRNO=5 if we try to write to the pty, +but we still get no errors after that if we try to read from it. Still, +prior to this we did more than 20 unproductive nonblocking reads from the +pty (no error, no bytes) without incident; there did not seem to be anything +waiting. In fact, the last thing we read from the pty were the text +messages that are issued at the end of the transfer: "rz 3.73 1-30-03 +finished." After which it pauses a second and spits out a message about +UNREGISTERED COPY. + +Figured out how to build lrzsz, in hopes that the previous problems were +with rzsz and crzsz's fiddling with file descriptors, but I get the same +behavior. Which is good, I guess, because if I can fix one, I fix them all. +Or not... Testing lrz by itself (not under C-Kermit), I see that it doesn't +work at all with Kermit's own Zmodem implementation. + +OK, here's one problem: at the end of the transfer, the Omen Zmodems print +stuff like "Please read the license agreement", Kermit dutifully reads this +from the pty and sends it to the host, the host shell says "Please: command +not found", issues its prompt again, which Kermit reads, feeds to the pty, +and apparently the pty echoes it, so we send it back to the host, and there +ensues an infinite loop of getty babble until the pty closes. Now, there +ought to be a way to make the external protocol shut up, like Kermit's +-q(uiet) flag, but these are unregistered versions so you can't shut up the +messages. In fact, the transfer works, but the getty babble at the end +ruins the experience. Now I'm beginning to wonder how any of these programs +ever worked as external protocols. Hmmm, now that I try it, I see the +same thing happens the old way, when using ttruncmd() rather than ttptycmd(). + +Reading the crzsz documentation I see it says that messages come out on +stderr. OK, that's progress. In ckupty.c I try redirecting 2 to /dev/null. +Well good, this filters out the messages from csz, but we still get getty +babble on the prompt. In the debug log, we read the last bunch of stuff +from net, 618 bytes of Zmodem stuff... Now what happens? + +Zmodem on the remote exits, the host prints its prompt. Kermit, of course, +reads the prompt from the net, now come to the bottom of the loop and we +have 7 bytes to write to the pty, and no error condition, so we continue the +loop. select() says that the pty is ready for writing. We write the 7 +bytes and and get no error. Loop again, this time select() says the pty has +data waiting. Sure enough we get the prompt back, and send it to the net, +and thus begins the getty babble. There are two causes for this: + + 1. crzsz does not exit immediately; it sleeps for 10 seconds after + printing its nag message. + + 2. During this interval the pty seems to be echoing what is sent to it. + csz is not echoing; I checked. Anyway, removing the pause doesn't + seem to make a difference. + +ttptycmd() needs to: + + . TELL the pty module to redirect stderr to /dev/null + . SET PTY TO NOECHO (master or slave?) + +Tried setting the pty to noecho: + + termbuf.c_lflag &= ~(ECHO|ECHOE|ECHOK); + +and this seemed to stop the getty babble. After the file transfer, I read +back the prompt from the host shell, I write the prompt bytes to the pty; +there is no error. And now select() simply hangs forever (or times out if +a timeout is set). The question here is: why didn't writing to the pty +produce an error? And, because we never detect the pty has exited, we can't +set a good return code. 5 Jan 2007. + +Moved pty fork testing to a separate routine, pty_get_status(), and +added a call to it from the place where we time out, in case the fork +terminated; then we can get and return its status. 6 Jan 2007. + +Added calls to pty_get_status() to every place where we suspect a pty error, +tried again with lrzsz, crzsz, and regular rzsz. All three work, but in +each case waitpid() indicates that the sz program gave exit code 1 (failure). +ckutio.c, 7 Jan 2007. + +Changing the subject... On my test system, every time I execute ttptycmd(), +I get "permission denied" on /dev/ttyp3. Then I run it again and get to +ttyp4 which is OK. I wanted to skip past any pty for which I lack +permission and try the next without raising an error. Added debugging code: + + 16:25:23.524 pty_getpty() pty master open error[/dev/ptyp0]=5 + 16:25:23.524 pty_getpty() pty master open error[/dev/ptyp1]=5 + 16:25:23.524 pty_getpty() pty master open error[/dev/ptyp2]=5 + 16:25:23.524 pty_getpty() found pty master[/dev/ptyp3] + 16:25:23.524 pty_getpty() slavebuf [2][/dev/ttyp3] + +So it already was skipping past open errors; ttyp3 was opened successfully. +The problem is that ptyp3 is rw-rw-rw-, but the corresponding master, +ttyp3, is rw--r----. It seems the code assumes that if the master can be +opened, then so can the corresponding slave. Unfortunately, the code is +not structured to allow us to skip ahead to the next master if the slave +can't be opened. 7 Jan 2007. + +Spent a couple hours trying to rearrange the code in the pty module to skip +past inaccessible slaves but it was a rabbit hole, not worth it, backed off. +8 Jan 2008. + +Tried an upload over a secure connection using lsz. Unexpectedly, this time +it worked; not only was the file (about 0.5MB) transferred correctly, but +Kermit detected the fork's termination and got the pid's exit status, and, +for the first time, correctly reported a successful transfer. I have no +idea why this works today and not yesterday. More tests; it works most of +the time. It works with csz and with regular sz too. + +(days later...) + +ckucns.c seems to do the right thing; it recognize the ZSTART string, +activates the Zmodem-Receive APC, and returns. doconect() sees the APC and +begins to execute it. The RECEIVE command results in a call to the GET +command parser, doxget() (IS THAT RIGHT?), then comes a ttflui(), which +throws away a bunch of stuff. Finally we get to ttptycmd(), we get a pty +and run lrz in it, select() says stuff is waiting from the pty, but read +returns 0, errno 0. Skipping the ttflui() in doxget() if the protocol was +not Kermit didn't seem to make difference. ckuus6.c, 8 Jan 2007. + +The problem is that in this case, reads from the pty never get anything (no +data, no error), write always gets an error. It's as if the pty was not +being set up right, or we're using the wrong file descriptor. And if we +skip the autodownload? Same thing. + +OK, putting downloads aside for a moment, let's get uploads working as well +as possible. At this point we have the odd situation (at least in this +configuration) that the upload succeeds, but now for some reason we are +unable to read the exit status from the process, even though this was +working before, so ttptycmd() returns 0 (failure), yet Kermit reports +success. + +Well, it turns out that kill(pty_fork_pid,0) was gumming up the works. +If we use only waitpid() all is well, I think. waitpid() with WNOHANG +returns -1 with status -1 errno 0 if the pid has not exited, and it returns +the pid and status > -1 if the process has exited. Fixed pty_get_status() +to do it this way. ckutio.c, 7 Jan 2007. + +Let's move this from Mac OS to NetBSD and see how it works. Well, the file +transfer was just fine, but then I used some sexps to calculate the elapsed +time and transfer rate, and Kermit hung in dosexp(). Fine, ignoring that... +The debug log shows that ttptycmd() gets the pty OK, master and slave, the +i/o goes smoothly, and waitpid() does its job perfectly. Solaris, same +deal; ttruncmd() goes smoothly, but then the sexps afterward get "Arithmetic +exception". Turns out there was a BAD bug in dosexp() that allowed an +integer division by 0 to occur under certain circumstances; it's always been +there. Fixed in dosexp(): ckuus3.c, 8 Jan 2007. + +After noticing a few problems running the pop.ksc script in production over +the past year, rewrote \femailaddress() to be more reliable and a lot +simpler. ckuus4.c, 9 Jan 2007. + +Back to ttptycmd()... When we left off, we could send but not receive. Set +up a test case using Kermit as the external protocol for receiving a short +file. If I SET STREAMING OFF and use short packets, it actually does work, +so it's not a complete failure to function, but apparently a lack of flow +control for the pty. Began by completing the parameterization of the pty +module, so it can be called for interactive use (fc 0) or for running +protocols (1). Confirmed that everything works at least as well as before +(e.g. "set host /pty emacs" vs external protocols). ckcdeb.h, ckutio.c, +ckupty.c, 9 Jan 2007. + +Found in HP-UX "man 7 pty" a description of ioctl(fd,TIOCTTY,fc) which is +exactly what we want: fc 0 turns off all termio processing and guarantees an +uninterrupted, unmolested, flow-controlled stream of bytes in both +directions. This function also exists in Linux, but not in Solaris, NetBSD, +or Mac OS X (TIOCNOTTY is not what we want, it does something else entirely). + +Another possibility is TIOCREMOTE, which "causes input to the pseudoterminal +to be flow controlled and not input edited, regardless of the terminal +mode". This one exists in at least HPUX, NetBSD, Solaris, and Mac OS X. + +Solaris: builds OK, but at runtime we get ENOTTY ("Inappropriate ioctl for +device"). By the time this happens, it's hard to tell from the code whether +the fd we're using is for the master or the slave; TIOCREMOTE can be used +only on the master. Close inspection shows that I am indeed doing that; +ptyfd as seen by ttptycmd() is truly the master, i.e. the /dev/ptyXX device, +not the /dev/ttyXX device (the slave fd can't be seen at all, as it exists +only in a separate fork). OK, so now we know that TIOCREMOTE can't be used +on Solaris. + +NetBSD: Somehow, whether as a result of today's fiddling or the phase of the +moon, the code in pty_open_slave() that tries to open /dev/tty started +failing on NetBSD ("Device not configured"). Changing it to be run only if +fc == 0 (which doesn't seem to hurt anything), once again I get ENOTTY on +the TIOCREMOTE ioctl. Zmodem works but Kermit totally fails (the fork exits +immediately with an exit code of 0, even though it didn't do anything). + +Mac OS X: Exactly the same sequence and results as NetBSD. + +Linux: It did not execute the new ioctl at all; apparently the TIOC symbols +are hidden or not exported or something. + +Where we stand: + . Downloads don't work + . Uploads got slow again + . Kermit doesn't work at all as an external protocol + +Actually if I take the debugging out it goes fast, but it doesn't finish. + +All today's work on ttptycmd() looks like a dead end. To roll back to +yesterday: + + cp ckutio.c-20070108 ckutio.c + cp ckupty.c-20070108 ckupty.c + cp ckupty.h-20070108 ckupty.h + +or to continue with today's: + + cp ckutio.c-20070109 ckutio.c + cp ckupty.c-20070109 ckupty.c + cp ckupty.h-20070109 ckupty.h + +Comparing Monday's and Tuesday's pty-related code, the differences are: + 1. Passing of function code to and among pty modules. + 2. Skipping the TIOCSCTTY ioctl and the open("/dev/tty") test. + 3. Attempting to put pty in TIOCTTY or TIOCREMOTE mode. + +Commenting out 2 and 3 should put us back where we were on Monday if the +parameterization was done right. And with this, on Solaris, downloading +with Kermit external protocol works but slowly, 8K cps, with or without +debugging. Debug log does not show any obvious bottlenecks; select() takes +anywhere between no time at all and 0.1 seconds to return. If I increase +the pty-net buffer size from 1K to 4K, the rate goes up to 55K cps. If I +make it 8K I get 136K cps. With 16K I get 346K cps. 32K: 395K cps -- this +last one isn't worth the doubling. But at 24K I get 490K cps, sometimes +twice that. Let's stick with 24K for now. Downloading with Zmodem (rzsz) +works at the same rate, but now we're back to seeing the getty babble +(Several "**B0800000000022d") at the end. 10 Jan 2007. + +Moving to Mac OS X, everything works the same as on Solaris, except I don't +get the Zmodem getty babble there, not even with Omen rzsz. Tested sends +in both remote and local mode, the latter over a secure Kerberos 5 Telnet +connection, using C-Kermit, rzsz, lrzsz, and crzsz, all good. 10 Jan 2007. + +Now we're back where we were yesterday morning, but with better throughput. +The big issue then was receiving files. But yikes, now it works! Not only +that, I got a transfer rate of 2.1M cps. That's using Kermit protocol, +streaming, and big (4K) packets. Which didn't work before. Not a fluke +either, I uploaded bigger and bigger files up to 6MB, they all went +smoothly, at rates between 1 and 2 MBps. 10 Jan 2007. + +Not so great in Zmodem land, however. If I start the external-protocol +receiver on the far end, escape back and start a Zmodem send... nothing. +If I leave the remote C-Kermit at its prompt (where it supposed to recognize +the Zmodem start string), still nothing. On the other hand, if I do it +with a script instead of by hand: + + def xx output take blah\13, send /proto:zmodem \%1 + +it works, at least intermittently. But that's in remote mode. We won't be +using this in remote mode. In local mode, where we have a secure connection +to another computer, it seems we can read from the pty and write to the net, +but we time out waiting to read from the net; nothing arrives. Well, we +know that i/o works both ways, so there is some kind of screwup with the +Zmodem protocol start itself. Increasing the (still hardwired timeout) from +5 to 22sec and driving the whole process with a script so as to avoid +autodownload as well as manual dexterity effects... It just sits there +forever, way longer than 22 sec. ^C'ing out, I see that sz was indeed +started on the far end and the protocol was executing. But it looks like +the receiver (the one running under ttptycmd()) is getting trashed packets, +because (a) it seems to be sending the same thing over and over again, and +(b) sometimes it waits as long as 10 seconds before anything arrives from +the remote. Maybe I was too impatient; I interrupted it after 4 minutes but +it seems to have been making some progress. Whenever there was data +available to read from the net, it was always 65 bytes, and it was not +actually the same data over and over. This is using lrz as the external +protocol. crz gets a bit farther. In this case we read up to 24K at a +gulp, but the amount varies a lot. It looks like we took in about 1.2MB of +Zmodem protocol data, but were only able to output the first 20K of the +file. Clearly there were lots of errors. In the end, the crz exits with +status 1 (failure). + +Anyway it looks like we're back at needing to find a way to accomplish +something like TIOCREMOTE on the pty, which is where we came in. 10 Jan 2007. + +Without any way to make the pty transparent and flow controlled, it would +seem to make sense to write to the pty in smaller chunks than we do to the +net. I left the read-from-pty-write-to-net buffer at 24K and changed the +read-from-net-write-to-pty buffer to 48 bytes. + +Upload using lsz worked but took about 3 minutes. Actually it didn't work. +On the local end it seemed to work, but the file did not appear on the +remote end. Tried this several times, each time with different results, +adding more debugging each time. The problem this time was that the pty +read could get EWOULDBLOCK. Changed the code to not treat this as an error, +now Zmodem uploads are solid again except I never got EWOULDBLOCK again +either, even though I repeated the same upload about 1000 times (with +throughput of over 2MBps even with debugging on), so the test for it has +not been exercised. + +OK, uploads still work. Back to downloading... The very first pty read +gets 0 bytes, followed by the fork test that shows that it exited with +exit status 2. + +Next we try starting sz with some different options on the far end: + + -q: quiet (no messages): + for some reason this gets totally stuck. + it looks as if this option is misdocumented; + sz seems to be sending the letter C (as in Xmodem 1K or whatever) + + -e: escape (all control chars): + first attempt to read pty finds the process gone with exit status 2. + + -k: send 1k blocks: + this one didn't stop immediately. It reads 48 bytes from net, writes + 48 to the pty with no error. Then reads 21 bytes from the pty, writes + them to the net OK. Then reads 48 bytes from net, writes them to pty OK, + reads 21 from pty, writes to net OK, etc etc... It appears to have + worked but (final read from pty returned 0, fork test showed lrz exited + with status 0), but only 754 bytes were received from the net when the + file is 420K... + +Well this only goes to show that the faster we shove stuff into the pty, the +worse it gets. Zmodem downloads won't work unless we can make the pty +transparent and flow-controlled. So to summarize today's developments: + + . separated in/out buffer sizes + . handled EWOULDBLOCK + . found out that sz options don't help much + +11 Jan 2007. + +Next day. This has got to be the most delicate code ever, it's like +Whack-A-Mole, fix A and B pops up. Even without touching it, something that +worked perfectly a 2:00 doesn't work at all an hour later. Maybe I could +have used pipes instead of ptys, but pipes have problems of their own. +There has to be a way to do this. The telnet server, the SSH server, etc -- +they all run on ptys, and we can upload files to them with Kermit. Why? +Because Kermit puts its terminal into all the right modes using the +time-honored methods of ttpkt() and ttvt(). Perhaps all we need is a copy +of ttpkt() that operates on the pty. + +On that theory, let's go back to Kermit as the external protocol. +It's important to suppress all messages and displays. With that, +uploads work fine, no hitches. + +Downloads: We fail right away. The debug log shows the Kermit program that +we are starting in the pty says: + + "" - Invalid command-line option, type "kermit -h" for help. + +But of course we are not giving it an invalid command-line option. +Switching to gkermit for the external protocol, now we see that no matter +command-line options we use, we read 0d 0d 0a from the pty and then the +next time we go to read from the pty we get 0 bytes and waitpid() says the +program has exited with status 1. + +Why should downloading be different from uploading? ttptycmd has no idea, +it does everything the same. The only difference would seem to be which +side sends first, but even that tends to get washed out by each program's +startup messages. + +Downloading with Kermit worked 2 days ago, what's different now? The buffer +sizes. Putting the net-to-pty back up to 24K (from 48 bytes)... Now it +works again. + +Conclusion: Kermit conditions the pty correctly, Zmodem does not. Therefore +ttruncmd() must duplicate what ttpkt() does. + +Or not. Because rz works fine on ssh/telnet ptys too. But not on our pty. +lrz exits immediately with status code 2 = 01000 but there are no clues in +the lrz.c source code, I don't even see this exit status set anywhere. +Unredirecting stderr, I see that the error is "lrz: garbage on command line". + +Why do both Kermit and Zmodem sometimes think they are receiving an invalid +command line? If I could capture the garbage... + +Side trip #1: ("pty.log",O_WRONLY) gives "no such file or directory". +Changed this to ("pty.log",O_CREAT,0644) and now it doesn't get an error, +and it creates the file, but not with 0644 permissions, and with nothing +written in it. How come nothing works? + +Fine, the debug log shows that ttptycmd() receives the correct string +(e.g. "lrz -v"). It passes it to do_pty() correctly, and do_pty() passes it +to exec_cmd(), which runs cksplit() on it, coming up (in this case) with +"lrz" and "-v", which is right, and then: + + args = q->a_head + 1; + execvp(args[0],args); + +execvp() wants the args array to have a null element at the end. cksplit() +does indeed do that, or at least the code is there. Added code to exec_cmd() +to verify the argument list and that it is null-terminated. So far it is. + +Anyway, we have traffic between the Zmodem partners, but no joy. +Commenting out the bit that redirects stderr, now I can see it on my screen +in real time: + + lrz waiting to receive.Retry 0: Bad CRC + Retry 0: Got ERROR + Retry 0: TIMEOUT + Retry 0: TIMEOUT + Retry 0: TIMEOUT + Retry 0: TIMEOUT + +etc etc, forever. Trying sz -e on the far end, I get: + + Retry 0: Bad CRC + Retry 0: Garbage count exceeded + Retry 0: Garbage count exceeded + Retry 0: Garbage count exceeded + Retry 0: Garbage count exceeded + Retry 0: Garbage count exceeded + Retry 0: Garbage count exceeded + Retry 0: Garbage count exceeded + ... + Retry 0: Got ERROR + Retry 0: Bad CRC + Retry 0: Got ERROR + Retry 0: Got ERROR + lrz: xxufio.c removed. + +So apparently it's not a matter of escaping. Trying some other stuff, I +caught the command-line problem in the act: + + lrz: garbage on commandline + Try `lrz --help' for more information. + +Debug log shows: + + cksplit result[lrz]=1 + cksplit result[-v]=2 + exec_cmd arg[lrz]=0 + exec_cmd arg[-v]=1 + exec_cmd arg[]=2 + +An empty string at the end instead of a null pointer. I really do not see +any way that could happen, but rather than dig into cksplit() again after +all these years I added a test for this in exec_cmd(), which, of course +after adding it, never encountered this behavior again. + +Fiddled with pty buffer size again. Made it 512 bytes instead of 24K. +Zmodem downloads are the same (Rety 0: TIMEOUT, over and over). But I don't +see what the problem is -- every time we receive n bytes from the net, we +write n bytes successfully to the pty and there are no errors. But it also +looks like the remote sender is sending the file header over and over +because it's not receiving an acknowledgment. If we're not losing data, +then maybe it's a transparency problem. + +Tried uncommenting the TIOCblah stuff I commented out before. Now instead +of only timeouts I get: + + lrz waiting to receive.Retry 0: Bad CRC + Retry 0: Got ERROR + Retry 0: Bad CRC + Retry 0: Got ERROR + Retry 0: Bad CRC + Retry 0: Got ERROR + Retry 0: TIMEOUT + +which is odd because the TIOCREMOTE ioctl failed with errno 14, EFAULT, +bad address, which should indicate it had no effect. We're still receiving +data from the remote in tiny chunks (from 12 to 65 bytes), apparently the +same stuff (file header), and writing them to the pty successfully but +nothing... + +Looked at cloning ttpkt() for the pty, but these stupid routines use global +tty mode structs so it's not going to be easy. + +Well, we got exactly nowhere today, but I think I'll leave stderr as it is +so users will see some feedback; no reason not to. + +WHY DO KERMIT DOWNLOADS WORK AND ZMODEM NOT? + +Is it 8-bit transparency? Up til now I've been testing with text files. +If I try to download a binary what happens? Fails after 99 seconds. Packet +log from the far end shows that as soon as the first packet containing 8-bit +data is sent, everything stops. At least I got one of these: + + 17:23:56.475 exec_cmd arg[gkermit]=0 + 17:23:56.475 exec_cmd arg[-qr]=1 + 17:23:56.475 exec_cmd arg[]=2 + 17:23:56.475 exec_cmd SUBSTITUTING NULL=2 <-- the code I just added + +Doing this again shows the same thing on the near end. All the 7-bit-only +packets are sent and acknowledged OK. Three 8-bit data packets arrive and +nothing else happens after that. This is with G-Kermit. + +The same thing happens with C-Kermit receiving. But if I change C-Kermit's +.kermrc to turn off streaming and use a short packet length: + +The transfer works, even though it's sending 8-bit bytes. So the problem is +not 8-bit data after all, per se. Facts: + + . Kermit can receive streaming transfers of 7-bit files. + . Kermit can not receive streaming transfers of 8-bit files. + . Kermit can receive nonstreaming transfers of 8-bit files with short packets. + . Kermit can receive nonstreaming transfers of 8-bit files with 1K packets. + . Kermit can receive nonstreaming transfers of 8-bit files with 4K packets. + +So it's the combination of streaming and 8-bit data? 12 Jan 2007. + +As a test I made a new routine pty_make_raw() that does cfmakeraw() (a +nonportable "POSIX-like" function known to be used on ptys in applications +that do approximately what we're attempting). Results: + + Solaris: errno 25 - inappropriate ioctl for device. + +This happens even when we try to get the terminal modes with tcgetattr(), +which is completely nuts. We pass it the file descriptor of the pty master, +which is supposed to work. But in Mac OS X, there are no errors. But +downloads still don't work; lots of errors but the pattern is different. +Using a very small buffer: + + Retry 0: Bad CRC + Retry 0: Garbage count exceeded + Retry 0: Garbage count exceeded + Retry 0: Got TIMEOUT + Retry 0: TIMEOUT + Retry 0: Bad CRC + Retry 0: Bad CRC + Retry 0: Garbage count exceeded + Retry 0: Garbage count exceeded + Retry 0: TIMEOUT + Retry 0: Got ERROR + Retry 0: TIMEOUT + Retry 0: Garbage count exceeded + Retry 0: Garbage count exceeded + Retry 0: Bad CRC + +Using a bigger buffer: + + Retry 0: Garbage count exceeded + Retry 0: Garbage count exceeded + Retry 0: Garbage count exceeded + Retry 0: Garbage count exceeded + Retry 0: Garbage count exceeded + Retry 0: Garbage count exceeded + (several screensful) + +Various other combinations... Nothing seems to work. + +Insight: telnetd does exactly what we want to do, sort of. +But it uses TIOCPKT, so every time it reads from pty, it receives +one control byte and then the data bytes, which would complicate our +buffering scheme considerably. Anyway the TIOCPKT ioctl() fails on +Mac OS X with 14 "Bad address". + +Also see: snoopserver.c (found in Google). It seems to do things in a +slightly different way -- it sets stdout to raw and then dups it to the +slave side of the pty? + +Maybe it's a mistake to use the ckupty.c routines. They are designed for +creating and accessing an interactive session. Maybe just copy one of the +other programs. + +18 Jan 2007. Tried going back to blocking rather than nonblocking reads +to see if it would make a difference, after all the other changes. Nope. +OK, let's look at some of these other programs... + +snoopserver.c. I don't know exactly what this is or where it's from or what +platform it runs on and there are no comments to speak of, but it does +approximately what ttptycmd() does. To get a pty it uses openpty(): + + if (openpty(&pty, &tty, NULL, NULL, NULL) == -1) + +then creates a fork. In the fork, it closes the pty (master) and +manipulates the modes of the tty (slave), dups tty to be stdio, and then +doex execv() on the command. Meanwhile the upper fork closes the tty +(slave), gets the attributes of stdin, using atexit() to have them +automatically restored on exit. Then it sets stdin to raw mode and enters +the select() loop on stdin, the pty master, and the net. It uses regular +blocking reads. It does not use TIOCPKT or anything like it. + +openpty() is supported on: Linux, Mac OS X, NetBSD, FreeBSD, ... +openpty() is NOT supported on: Solaris, HP-UX, ... + + 1. Try copying the pty code, but keep everything else the same. + +I did this; it compiles and starts OK, upper fork (ttptycmd) debug log shows +no errors, but nothing happens. Logs show that the Kermit program that is +started in the subfork seems to die as soon as it reaches eof on its init +file. The good news, at least, is that select() doesn't report report that +the pty is ready to be read. Clearly the file descriptors aren't being +assigned as expected, or as before. + +In ckupty.c getptyslave() dup2's the slave fd to 0 and 1. The new code +does exactly the same thing. Debug log makes it look like the forked kermit +is not receiving its command line. But now I'm not even sure that the +forked kermit started at all. ps from another terminal doesn't show it. + +19 Jan 2007: Noticed that in snoopserver, the select() calls use standard +input and output file descriptors, rather than the pty master. Made that +change... In doing that I had to look at every file descriptor in every +line of code and discovered a couple mistakes, fixed them, put back the +original code but with the fixes, tried it, but no change; can upload OK but +still can't download with Zmodem without lots of errors and ultimate +failure. Going back to the alternative version and trying to get the the +file descriptors sorted out, now it appears that the external Kermit program +never even starts in the lower fork. After a bit more fiddling I sort that +out, but now when the lower Kermit program goes to open "/dev/tty" it gets +errno 6 "Device not configured". Forcing it to use stdio with "-l 0", it +gets past this and actually sends its first packet. But the Kermit on top +reads nothing from the pty. + +Next, I change the pty fd from STDIN_FILENO and STDOUT_FILENO to slavefd. +No difference. Next I comment out the dup2() calls. This time I get some +action. The transfer starts, but only one packet comes. Log shows that +the lower Kermit sends its S packet. The upper Kermit receives the ACK +but the lower Kermit never gets it. The write to the pty succeeds, no +error. Different combinations give different results. If write to master +and read from the slave, I get packets in both directions but tons of +errors.... This happens only if I comment out the dup2()'s. + +25 Jan 2007: After leaving it sit for a while, and realizing that what I'm +trying to do has to be possible because so much other software does the same +thing (e.g. Telnet servers), I put things back to how they were originally +-- the upper fork (Kermit) uses the master and the lower fork the slave. +The upper fork puts the master in raw mode, the lower fork puts the slave in +raw mode. The lower fork dup2's the slave fd to stdin/out. Send file in +remote mode using external Kermit: works OK but select() times out at the +end. This means that the self-contained pty code in ttptycmd() is sorted +out -- all the file descriptors go to the right place, etc, and now we can +use this routine as a testbed, rather than the original ckupty.c-based one. + +But send with lsz, csz, and regular rz: Nothing happens, times out after 0 +bytes of i/o. Once again, Kermit works, Zmodem doesn't. The reason for +running Zmodem in a pty is so its i/o will work as it does on a terminal, +no matter how it may fiddle the file descriptors. So why don't we see a +single byte come out? + +Commenting out pty_make_raw(), I get a successful Zmodem send using lsz. +csz manages to get the filename across, but then gets stuck. regular sz, on +the other hand, works perfectly. Testing csz by itself (not under Kermit), +I see it fails in exactly the same way ("Got phony ZEOF", etc). OK, forget +crzsz. + +OK, let's move to local mode over a Kerberized Telnet connection... +Uploading (sending) with external Kermit protocol... works. +Downloading (receiving) with external Kermit protocol... works. +Uploading with sz... works. +Downloading with lrz... Gets tons of errors and fails. + +Running pty_make_raw() on the slave but not on the master: no difference. +Running pty_make_raw() on the master but not on the slave: no difference. + +Back where we started... Either: + + . Zmodem is overdriving the pty, no matter what modes we put it in. + . It's a transparency problem. + +Theoretically we should be able to test these by using different sz switches: + -q: quiet (should always use this) + -e: escape all control characters + -B n: Buffer n bytes (rather than whole file) + -L n: Packet length + -l n: Frame length (>= packet length) + -w n: Window size + -4: 4K blocksize (doesn't help) + +-q by itself doesn't help. +-q -e, this one worked but still got about 100 errors and was very slow. +-q -e -l 200 -L 100, failed fast and bad. +-q -e -w 1. Failed quickly. +-q -e -w 1 -B 100. Eventually failed. +-q -w 1, Eventually failed. +-q -l 1024, this gets much more errors, definitely need -e. +-q -e -l 1024, got pretty far before failing. +-q -e -w 1 -l 1024, also got pretty far before failing. +-q -e, this one got farthest of all, about 48K, before getting errors. + +In the latter combinations that work somewhat better, we always get up to +16K, or 32K, or 48K, before the errors start coming out and piling up. +Sometimes the errors are recoverable and we receive as much as 300K +successfully before giving up. + +Now that we have data flowing pretty well (but not well enough), tried +reinstating pty_make_raw(), but it hurt more than helped. + +As a sanity check, I tried transferring from the same host over the same +kind of connection (Kerberized Telnet) directly to K95's built-in Zmodem +protocol, and that worked fine. So the problem is definitely in the pty. +Or more precisely, where Kermit writes incoming net data to the pty master. + +26 Jan 2007: Tried changing the size of the net-to-pty buffer from 24K to +1K. Result: total failure. Set both buffers to 1K. Still total failure. +Set both to 4K: now we get about 45K of data, then failure. Put them both +back to 24K, still fails totally -- the same code that worked pretty well +yesterday. Actually, no downloads work, not even Kermit, not even of +text files. + +27 Jan 2007: Since I have not been able to find a way to make ptys work for +this, I made a third copy of this routine, this time using pipes instead of +ptys. The disadvantage here is that if the external protocol does not use +stdio, the pipes won't work, but one thing a time... + +Inferior Kermit starts in lower fork, but when it tries to send its first +packet it gets errno=9 EBADF, Bad File Descriptor. Substituting G-Kermit as +the external protocol, which is simpler, reveals that the problem is that +the external protocol gets errors when it tries to manipulate the its stdio +file descriptors with ioctls, etc; these are not valid for a pipe. The pipe +mechanism itself works. If I take out the test for ttpkt() failing in +gkermit, the file transfer works OK. Trying Zmodem... Sending works OK; +receiving works a lot better than with ptys (it got 360K into the file +before failing). Making the buffers smaller, doesn't help. + +I'm starting to wonder if the problem might be in my buffering code, rather +than in the pty or pipe interface... Try making a version that does +single-character reads and writes. + +This one reads the first packet from the lower Kermit and sends it. It is +recognized by the other Kermit, which sends an ACK. We see the ^A of the +ACK, but then select() times out on the next character -- OF COURSE: because +at a lower level, it has already been read. We have to check the myread +buffer, and then call select() only if it's empty. Making this change: + + . SEND with G-Kermit works (but very slowly). + . SEND with lsz works but gets a lot of errors, eventually succeeds. + +Let's work our way back... With the same changes to the buffered pipe version: + + . SEND with G-Kermit/streaming works (fast). + . SEND with lsz works too (fast), but we get gubbish at the end. + . RECEIVE with Kermit fails because "/dev/tty is not a terminal device". + . RECEIVE with rsz... lots of errors ("garbage count exceeded") but succeeded. + +But maybe now we're seeing pipe artifacts, so going back one more step to +the version that gets its own pty and starts its own fork: + + . SEND with G-Kermit/Streaming works (fast) but select() times out at the end. + +Another breakthrough: Moved the write pieces to below the read pieces. This +is what was preventing the buffer reset code from working -- with the writes +done before the reads, we never catch up and can never reset the buffers. + + . SEND with G-Kermit/streaming works (fast) (but there's a pause at the end) + . SEND with lsz works (fast) (but there's a pause at the end) + . RECEIVE with rsz... lots of errors ("garbage count exceeded") and fails. + . RECEIVE with Kermit -- nothing happens (it thinks it succeeded), then we + reconnect, terminal sees S packet and goes into autodownload + +From the log it looks like ttpkt() fails in the lower Kermit. Switching +this with the hacked G-Kermit... it gets "transmission error on reliable +link". Tried again with real Kermit below, this time with "-l 0" and not +streaming. This was actually working, but slowly, I don't see any NAKs in +the packet log, but then select() timed out. + +28 Jan 2007: Restored both the calls to pty_make_raw(): + + . SEND with C-Kermit streaming works, but slow (54Kcps) + . Ditto, but with debugging off -- hangs forever. + . Ditto, but using G-Kermit instead of C-Kermit -- also hangs forever. + +Backed off on calling pty_make_raw(). Same thing. +Reduced size of net-to-pty buffer. Same thing. + +15 Feb 2007... Decided to give up on this and publish it as is, in hopes +that somebody with more experience with ptys can make it work, because I'm +just going in circles. So today I just have to get the code into shape so +people could choose among the three alternative routines. The second one, +yttyptycmd(), is the one that uses openpty(), which is not portable, so it +can be enabled only for Mac OS X, NetBSD, FreeBSD, and Linux, or by also +defining HAVE_OPENPTY at compile time. Anyway, if you build Kermit in the +normal way, you get the regular behavior -- ttruncmd() is used to execute +external protocols. If you build it with -DXTTPTYCMD, you get the first +version of ttptycmd(); with -DYTTPTYCMD the second, and with -DZTTPTYCMD the +third. + +(Then some interruptions, then...) + +From Jeff, fix hostname comparison in X.509 certificate checking to work +right in the case of names that contain no periods. dNSName_cmp(): cl_ssl.c, +21 Feb 2007. + +John Dunlap noticed some strange behavior when transferring files between +home base and the EM-APEX oceanographic floats via satellite... long story, +but every so often the transfer would get stuck for a long time, and it +happened only when C-Kermit was sending a file and received two or more +packets (Ack or Nak) back to back from the float. Years ago I added some +lookahead code to ttinl() to clear the input buffer of any interpacket junk +so that, in the windowing case, we wouldn't be tricked next time around into +thinking a packet was waiting to be read when there wasn't. The code, which +has been there for a while, was a bit fractured; luckily, it would be +executed only when the debug log was active so it didn't have much effect. +The problem was that if the SOP came immediately after the EOP, it could be +missed because the loop read the next character before checking the current +one. Fixed by rearranging the loop. Also I changed it so it would execute +in all cases, not only when the debug log was active. Also, cleaned up a +bunch of confusing #ifdefs and removed some chunks that had been commented +out for years, decades maybe. ttinl(): ckutio.c, 21-22 Feb 2007. + +Added NOW keyword info to HELP DATE, plus a tip about how to convert to UTC; +suggested by Arthur Marsh. ckuus2.c, 22 Feb 2007. + +When an FTP client sends NLST to the server and no matching files are found, +the server is supposed to respond with an error message on the control +channel and nothing on the data channel. However it seems that at least one +server sends the error message back on the data channel, as if it were a +filename ("/bin/ls: blah: No such file or directory"), and on the control +channel there is no error indication ("226 ASCII Transfer complete"). At +this point remote_files() has a listfile and, if a match pattern was given, +it looks through list to see if any of the lines match the given filename, +e.g. "blah". This makes FTP CHECK give false positives. The problem +(diagnosed by Jeff) is that the match pattern was not given in this case, so +it takes some random default action, resulting in the spurious success +return. Fixed by using the user's string as the pattern. Not tested, +however, since I don't have access to a server that behaves this way. +ckcftp.c, 22 Feb 2007. + +If an external-protocol file transfer fails, don't print Kermit-specific +hints. ckuus5.c, 22 Feb 2007. + +One more time with ttinl(). Got rid of the "csave" junk, which never could +have worked (which is no doubt why it was in a debugging section). The +problem was that saving the beginning of the next packet locally did not +synchronize with the buffer clearing (ttflui()) done at a higher level, +between calls to ttinl(). So now, the lookahead code, if it finds the +beginning an as-yet unread packet, puts it back at the head of the input +queue. This way, if the protocol engine clears the input buffer, it will +get the whole packet, not just the part after the SOH. ckutio.c, 24 Feb 2007. + +From Steven M Schweda, Saint Paul, MN: adaptation of large file support to +VMS (it was already possible to transfer large files in VMS C-Kermit but the +file-transfer display and statistics were wrong). And a minimal adaptation +of the FTP client to VMS -- no RMS, no special VMS file stuff, Stream_LF and +binary files only, developed and tested only with UCX. SSL/TLS is +supported. The source-code changes are minimal; most have nothing to do +with VMS, but with header files, prototypes, and data types (e.g. ftp_port +int rather than short, various signed/unsigned conflicts) to shut up +compiler warnings. Some of these could be dangerous in terms of +portability; I've marked them with /* SMS 2007/02/15 */. ckcfns.c, +ckcnet.h, ck_ssl.h, ckuus3.c, ckuus4.c, ckvfio.c, ckcftp.c, ckvker.mms +(which was rewritten to actually reflect the source module dependencies), +ckvker.com (also heavily modified). ckvker.com (the "makefile" for VMS +C-Kermit) now includes "F" and "I" option flags for the large File and +Internal ftp features, plus better handling of Vax/Alpha/IA64 distinction. +26 Feb 2007. + +Changed NetBSD targets to include -DHAVE_OPENPTY and -lutil, so they +can use openpty(). makefile, 26 Feb 2007. + +Built on Solaris without and with SSL OK. +Built on NetBSD with Kerberos 5, OK. +Built on Mac OS X 10.4, regular version, OK. +Built on Mac OS X 10.4 with SSL and Kerberos 5, OK. + +On VMS 7.2-1/Alpha with MultiNet 4.4A-X... + +'CC' 'CCOPT' KSP:ckuus3 +%DCL-W-TKNOVF, command element is too long - shorten + \CKUUS4.OBJ "'CC' 'CCOPT' KSP:ckuus4" "KSP:ckuus4.c KSP:ckcsym.h KSP:ckcdeb.h + KSP:ckclib.h" "KSP:ckcasc.h KSP:ckcker.h KSP:ckcnet.h KSP:ckvioc.h" +"KSP:ckctel.h KSP:ckuusr.h KSP:ckucmd.h KSP:ckuver.h" "KSP:ckcxla.h + KSP:ckuxla.h KSP:ckcuni.h KSP:ckuath.h" + +The new rule for ckuus4.c was too long. I removed one file from the +dependency list (ckcxla.h, which will probably never change again) and that +made it OK. Built Nonet and Net versions OK, but this is without the new +stuff. + +"make f" (large-file support) on VMS 7.2-1... +'CC' 'CCOPT' KSP:ckuus4 + if (CKFSEEK(fp,(CK_OFF_T)j,SEEK_CUR) != 0) { +........................^ +%CC-I-IMPLICITFUNC, In this statement, the identifier "fseeko" is implicitly +declared as a function. + +Ditto for ftello and fseeko in various other places, and then fseeko and +ftello come up up undefined at link time. + +The rule for ckcftp in "make i" (Internal FTP support) had the same problem. +I removed ckcxla.h from its dependency list too, but "utime" comes up +undeclared at compile time and undefined at link time. + +Verdict: neither one of the two new features can be used in VMS 7.2 or +earlier, but the code still builds OK if you don't ask for them. + +VMS 8.3 on IA64... Can't build anything: +%MMS-F-BADTARG, Specified target (WERMIT) does not exist in description file + +27 Feb 2007: Changed CKVKER.COM to keep all its dependencies but use a +shorter logical name (Steven M Schweda). The problem on VMS 8.3 is that MMS +now supports case-sensitive file systems, and so it can't find anything. +Workaround: bypass MMS (include "m" in P1). With this, "@ckvker.com ifm" +builds OK on HP Testdrive, but I can't test the new features since outbound +connections are not allowed there. As for fseeko(), ftello(), and utime(), +they simply are not available prior to VMS 7.3. It would probably be a good +idea to test for this in CKVKER.COM, but actually it is possible to install +newer C's and CRTLs on older VMS versions, so don't stand in their way. + +28 Feb 2007: With additional changes from SMS, and then some further +adjustments, I was able to build the FTP version on VMS 7.2-1. First I +tested it with GET of a binary file, but it transferred it in text mode. +After a few more attempts with PUT and GET, it crashed with "floating/decimal +divide by zero" in ckscreen, ckuusx.c line 27859. Of course, that's the +listing line, not the source line, and I don't have a listing. + +To get a listing, I deleted CKUUSX.OBJ and then did: + + $ make i "" "" "/LIST" + +Surprisingly, it recompiled everything. + +Anyway, the divide by zero happened in a section of code where the divisor +was not checked, but it was a section of code we should not have been +executing at all, since the file-transfer display was fullscreen, and this +was in the "brief" section. Anyway, I added the needed check. Again, it +recompiles everything. Maybe there's no MMS on grumpy -- right, there isn't. + +ANYWAY... Try to GET a binary file like this: + + binary + ---> TYPE I + 200 Type set to I. + get gkermit + ---> TYPE A + 200 Type set to A. + ---> SIZE gkermit + 550 gkermit: file too large for SIZE. + GET gkermit (text) (-1 bytes)---> TYPE A + +Anyway... "get /binary gkermit" downloads it, seemingly correctly (the byte +count is right). + +But "put /binary gkermit.;1" results in a 0-length GKERMIT file being sent. +Here's the debug log: + +FTP PUT gnfile[DISK$MSA4:[C.FDC.NET]gkermit.;1]=1 +ftp putfile flg[DISK$MSA4:[C.FDC.NET]gkermit.;1]=0 +zltor fncnv[DISK$MSA4:[C.FDC.NET]gkermit.;1]=-1 +FTP PUT nzltor[GKERMIT] +zfnqfp 1[DISK$MSA4:[C.FDC.NET]gkermit.;1]=675 +zfnqfp 2[DISK$MSA4:[C.FDC.NET]GKERMIT.;1]=31 +zfnqfp 3[DISK$MSA4:[C.FDC.NET]GKERMIT.;1]=31 +zrelnam result 2[gkermit.;1] +ftp sendrequest restart[DISK$MSA4:[C.FDC.NET]gkermit.;1]=0 +openi name[DISK$MSA4:[C.FDC.NET]gkermit.;1] +openi sndsrc=-1 +openi file number=2 +zopeni[DISK$MSA4:[C.FDC.NET]gkermit.;1]=2 +zopeni fp=0 +chkfn=2 +chkfn return=0 +zopeni fixed file format - using blk I/O +zopeni binary flag at open=1 +zopeni ifile_bmode=1 +zopeni binary=1 +zopeni RMS operations completed ok +openi zopeni 1[DISK$MSA4:[C.FDC.NET]gkermit.;1]=1 +ftpcmd cmd[PASV] +FTP SENT [PASV] +FTP RCVD [227 Entering Passive Mode (166,84,1,2,233,216)] +initconn connect ok +FTP SENT [STOR GKERMIT] +FTP RCVD [150 Opening BINARY mode data connection for 'GKERMIT'.] +doftpsend2 ftpcode[STOR]=150 + + Here is where the file is supposed to be read and sent but there is nothing + in the log between the "doftpsend2 ftpcode" line and the following line. + +rftimer status=1 +gftimer status 1=1 +gftimer status 2=1409025 +gftimer status 3=1409025 +gftimer s[0.000000] +zclose n=2 +chkfn=2 +chkfn return=1 +zclose ZIFILE RMS operations completed ok +ftp getreply lcs=0 +ftp getreply rcs=-1 +ftp getreply fc=0 +FTP RCVD [226 Transfer complete.] +ftp getreply[226 Transfer complete.]=2 +doftpsend2 ok=0 + +Everything is OK up until we go to send the file, then it behaves as if we +got EOF immediately and so closes the data connection, and reports success; +an empty copy of the file is left on the far end. + +Starting over with a text file.... PUT LOGIN.COM gets another divide by +zero. But it happened in the code I just fixed, which is impossible. Swell. +I recompiled everything and this time the upload worked, and downloading it +again worked too. + +But a binary file still can't be uploaded. Trying to upload a text file +after doing this seems to succeed (reports the right number of bytes sent) +but nothing appears on the far side. + +SUMMARY: + + To download a text file: GET /ASCII blah.txt (/ASCII is optional) + To download a binary file: GET /BINARY blah.bin (/BINARY is required) + To upload a text file: PUT blah.txt (/ASCII switch not needed) + To upload a binary file: PUT /BINARY blah.bin (doesn't work) + +Problems: + . Why doesn't BINARY "stick"? + . Why don't binary uploads work? + +The culprit seems to be the VMS version of zxin(). In the FTP module, +zxin() is called only when sending binary files. In VMS, zxin() is just +a front end for C-Library fread(). It probably needs to do just do +zminchar() in a loop, like binary mode does, but calling zzout instead +of xxout. Or something like that. FINISH THIS TOMORROW (debug on grumpy). + +2 Mar 2007: New logs from John Dunlap. + +ema-1636-log-0175.dbg: C-Kermit uploads a short file. It receives an Ack for +the Z packet it just sent, tailgated by the beginning of a Nak for the next +packet. When the second SOH is encountered, it is put back in the myread +queue. Then the protocol engine, to which we return the Ack, says, "I have +the packet I wanted so I'm clearing the buffer", and away go the first two +bytes of the Nak from the myread buffer. Then, having just received the Ack +of our Z packet, we send our B, and go to read the reply. in_chk finds 0 in +the myread buffer (which we just cleared) and 6 waiting to be read from the +comms channel, which it does, obtaining the remaining 6 bytes of the Nak, +which it properly discards. (The reason this is proper is that, having +already received the Ack for the last packet it sent, no Ack or Nak that +arrives subsequently -- in the non-windowing case -- could possibly affect +what it does next.) Since it hasn't yet found a good packet, it keeps +reading, and now it finds the Ack to the B, as soon as it showed up. This +is how it's supposed to work. No time was lost because of anything that +C-Kermit did. + +ema-1636-log-0174.dbg: C-Kermit uploads a short file. It sends Data packet +#3 and receives the Ack followed immediately by the first 3 bytes of a Nak +for packet 4. When it gets to the SOH of the second packet, it pushes it +back in the queue. Again, input() flushes the input buffer (myread queue +and device buffer). C-Kermit detects EOF on the file it is sending, and +sends the Z packet. Then it reads the remaining bytes of the Nak, +which it discards, and then it finds the Ack for Z which comes in 23 seconds +later, sends the B, gets a Nak for the B, sends the B again, gets the Ack +for the B 4 seconds later, and done. Again, it's working right and losing +no time. + +The question remains: what would happen if the protocol engine did not clear +the buffer? Would ttinl() retrieve all packets in sequence even when they +come back to back? To test this, I had C-Kermit send a file using 30 window +slots and observed the stream of Acks in the reverse direction: + + HEXDUMP: mygetbuf read (16 bytes) + 01 25 23 59 2f 52 39 0d | 01 25 24 59 2b 26 31 0d .%#Y/R9. .%$Y+&1. + ttinl lookahead my_count=9 + ttinl lookahead removed=^M + ttinl lookahead pushback SOP=^A + HEXDUMP: ttinl got (7 bytes) + 01 25 23 59 2f 52 39 | .%#Y/R9 + RECEIVE BUFFERS: + buffer inuse address length data type seq flag retries + 0 1 29212 9667 0 Y 3 0 + [%#Y] + ... + in_chk my_count=8 + ... + ttinl lookahead my_count=1 + ttinl lookahead removed=^M + HEXDUMP: ttinl got (7 bytes) + 01 25 24 59 2b 26 31 | .%$Y+&1 + RECEIVE BUFFERS: + buffer inuse address length data type seq flag retries + 0 1 29212 9667 0 Y 4 0 + [%$Y] + +Here we can see that the pushed-back SOH was properly retrieved next time +around, and the tailgating Ack was not lost. This scenario repeats itself +212 times in the log, and there are no screwups. + +Back to VMS FTP... The problem with sending binary files is that zxin() +uses C-Library fopen()/fread() instead of RMS, so it can't access the input +file, which was opened by zopeni(), which is totally RMS-ified in VMS +C-Kermit. For VMS only, I replaced the zxin() loop by a zminchar() loop +like the one used in text mode, except without the character set or +record-format conversion. Tested by PUT /BINARY of some binary files, which +worked fine. ckcftp.c, 2 Mar 2007. + +Next problem... VMS C-Kermit ftp client sending binary files in text mode. +Variation 1: We just send the file. zopeni() is supposed to detect that +it's a binary file and automatically set the mode. And it does: + + zopeni fixed file format - using blk I/O + zopeni binary flag at open=0 + zopeni ifile_bmode=1 + zopeni binary=0 + zopeni autoswitch from TEXT to BINARY + zopeni RMS operations completed ok + +but then in gnfile(): + + if (!server || (server && ((whatru & WMI_FLAG) == 0))) + binary = gnf_binary; /* Restore prevailing transfer mode */ + +Well, since VMS sets text/binary mode automatically when sending files, +this code can (and should) be skipped in VMS. gnfile(): ckcfns.c, 2 Mar 2007. + +Variation 2: BINARY or SET FILE TYPE BINARY doesn't force binary mode. But +SET FTP TYPE BINARY does. But BINARY does indeed call doftptyp() so what's +the problem? We do indeed set ftp_typ to 1 but it gets reset somewhere +before we call zopeni(). But then zopeni() puts it back to 1. Tracing +through a transfer, it looks like all of this works right, it's only that +the file transfer display says TEXT when the transfer is really in binary +mode. This is because screen() is called before openi(). I wonder if we +can call scrft() from the ftp module... No, that would be too easy. OK, +sendrequest calls openi() and sets the file mode; putfile() calls +screen(SCR_FN), which prints the transfer mode. But putfile calls +sendrequest() after it puts up the screen that says the file type. So it +looks like sendrequest() has to call screen(SCR_FN) again if it changes the +file type. OK, that did it. ckuusx.c, ckcftp.c, 2 Mar 2007. + +The BINARY and TEXT (ASCII) commands do not inhibit automatic type switching +in VMS. They don't in Unix either. They never have. Should they? I think +so, otherwise what good are they? Plus we want the Kermit FTP client to +behave like the others. I added code for this but it doesn't work, due to +the layers and layers of text/binary detection and switching and +if-this-but-then-if-that... Anyway, no harm done. The normal rule is: +when you PUT a file, Kermit figures out on a per-file basis whether to use +text or binary mode unless you include a /TEXT (/ASCII) or /BINARY switch +in the PUT (or MPUT) command. ckuus[r3].c, ckcftp.c, 2 Mar 2007. + +Wed Mar 7 16:21:13 2007 WROTE SHORT TEST PROGRAM for ttruncmd (the openpty +version) on Mac OS X. On dulce: ~/kermit/ttpty.c / ttpty.sh. It starts the +external protocol in the lower fork. The command to run is a command-line +argument. Sending and receiving files with Kermit works OK. But again, the +standalone program totally fails when I use sz or lsz as the external +protocol. So it looks like we can rule out any environmental effects of +running the code inside C-Kermit. + +Mon Mar 12 16:52:20 2007: Put some effort into making ttpty.c more useful; +added a debug log. Found that for some reason, at least on Mac OS X, +select() always timed out at the the end. I added a SIGCHLD alarm and that +seems to handle the fork exit condition very nicely. Now we can send (say) +a 3MB file at good speed on Ethernet (1Mcps) considering the debugging, etc, +and it terminates instantly. But when sending a file into ttptycmd (with +"gkermit -r"), things go wrong at the end -- the Z packet is never +acknowledged. This is reproducible. Maybe this is a good lead.... The log +shows that select() timed out, even though the gkermit fork had not yet +exited (or finished). It looks like gkermit sent the Ack, ttpty.c read it +from the pty and sent it out the net: + + 0003: read pty=8 <-- read Ack from pty + 0003: loop top have_pty=1 + 0003: loop top have_net=1 + 0003: FD_SET pty_in + 0003: FD_SET ttyfd in + 0003: FD_SET ttyfd out=8 + 0003: nfds=5 + 0003: select=1 + 0003: FD_ISSET ttyfd out + 0003: write net=8 <-- send ack to net + 0003: loop top have_pty=1 + 0003: loop top have_net=1 + 0003: FD_SET pty_in + 0003: FD_SET ttyfd in + 0003: nfds=5 + 0009: select=0 + 0009: select timeout - have_pty=1 + +But Ack never arrived. This is a streaming transfer. But nope, streaming +is not the problem. If I disable streaming ("gkermit -Sr"), we hang in in +the middle of sending the data. If I use small packets, we don't hang: +1000 is OK, 2000 is not. In fact, the cutoff is 1024. OK, TBC... + +Wed 14 Mar 2007: Receiving a file thru ttpty "gkermit -e 1200 -Srd" +produces a debug log that shows that gkermit gets a lot of EAGAIN errors +when it tries to read from its stdin. In fact, it takes 6 tries (read() +calls) to read the S packet (27 bytes). Then when the first data packet +arrives (1200 bytes), read() never returns even one single byte. The +timeout interval is 15 seconds and it times out repeatedly. Added a +primitive hex dump to the ttpty debug log for each read/write (showing only +the first 24 characters and the last character, so it fits on one line). +Tried uploading a file. The S, F, and A packets (short) are received and +Ack'd OK, but then ttpty select() times out, never receiving even one byte +from the D packet. Clearly, when the pty driver receives a burst of > 1K +bytes, stops working. As before, if I limit the packets to < 1K, it works +fine. + +Can I send an 8-bit binary file? Nope. ttpty reads the binary data just +fine from the net and writes it exactly as it was received to the pty, but +the first time we write an 8-bit byte, we never hear back from the PTY +again. But the log shows that when the initial 7-bit packets from the pty, +it looks like the PTY is not in rawmode, because these packets end with ^J +rather than ^M. Calling pty_make_raw() on the masterfd and slavefd +explicitly, however, doesn't change anything. It doesn't matter if I do +this in the lower fork or the upper fork. So maybe it's the actual +semantics of pty_make_raw() that are wrong. + +Thu 15 Mar 2007: Went thru all the terminal mode flags in Mac OS X; didn't +help. Changed hex dump to show whole packet. Put hex dump routine in a +private copy of G-Kermit. Tried to transfer an 8-bit file, logging both +ttpty and gkermit. Compared what ttpty received on stdin with what it sent +to the pty (same) and what was received by G-Kermit (same). Then I realized +that my little test program was not putting its controlling terminal into +raw mode; when I did that, I could upload binary files (streaming, 2MB/sec). +And with Zmodem too (with rz; lrz doesn't work for some reason). Looking +back at the original in ckutio.c, I see that ttptycmd() never called +ttpkt(). Maybe that was the trouble all along. (Yup, but maybe not the +whole trouble.) + +Moving back to C-Kermit and the original ttptycmd() routine, adding the call +to ttpkt(), and stripping out a lot of cruft, and moving the pty_make_raw() +code to ckupty.c, Kermit uploads and downloads (streaming) work fine in +Solaris. Zmodem sends a file, but then the transfer hangs at the very end, +as if the signoff protocol were lost. This happens on Solaris. If I move +back to Mac OS X, everything works just fine. Then, making a Kerberized +connection from the Mac to NetBSD, I can send files from the Mac with both +Zmodem and Kermit. Receiving... Kermit OK. Zmodem... Nope. "rz: +Persistent CRC or other ERROR" (and created a 265MB debug.log!) + +Fri 16 Mar 2007: ttptycmd() was for sending files with Zmodem across +encrypted connections. But it occurred to me that it's necessary for +clear-text connections too; e.g. Telnet, where 0xff has to be doubled. Of +course Zmodem doesn't do that itself, so there's no way Zmodem external +protocol could work when executed over a Telnet connection, and in fact +it doesn't. I wonder why I ever thought it did. + +Wed 21 Mar 2007: Back to where we left off a week ago. Trying C-Kermit's +ttptycmd() on the Mac again, in remote mode: + + . G-Kermit send txt (kst): OK 832Kcps + . G-Kermit recv txt (kr): OK 425Kcps + . G-Kermit send bin (ksb): OK 1000Kcps + . G-Kermit recv bin (kr): OK 188Kcps + +And Zmodem: + + . sz txt (zst): OK 563Kcps + . sz bin (zsb): OK 714Kcps + . rz txt (zr): OK 863Kcps + . rz bin (zr): OK 198Kcps + +So in remote mode, everything works. Now let's try a clear-text Telnet +connection... + + . G-Kermit send txt (kst): OK 841Kcps + . G-Kermit recv txt (krt): OK 391Kcps + . G-Kermit send bin (ksb): OK 811Kcps + . G-Kermit recv bin (krb): OK 171Kcps + +And Zmodem over the same clear-text telnet connection: + + . sz txt (zst): OK 91Kcps (*) + +Kermit is sending sz messages like "sz 3.73 1-30-03 finished." to the +host, which tries to execute them, after the transfer is finished. +Of course "sz" is a command, but: + + sz: cannot open 3.73: No such file or directory + sz: cannot open 1-30-03: No such file or directory + sz: cannot open finished.: No such file or directory + +Did I lose that code that dis-redirects stderr when I went back to using the +pty code from the ckupty module? No, it's there and it's being executed. +Apparently the copy of sz I have is writing its "finished" message to stdout +because "sz blah 2> /dev/null" does not suppress it. Starting again with +lsz instead of sz: + + . sz txt (lzst): OK 413Kcps + . sz bin (lzsb): OK FAILED (*) + . rz txt (lzrt): OK + . rz bin (lzrb): OK + +(*) Sigh. Using lsz, we get "garbage count exceeded" errors and eventual +failure. But using regular sz, we get the extraneous message that starts +sz on the far tend, and the resulting getty babble. + +But even without changing the code, it will work one minute, and then fail +consistently the next. For example, I was able to send files with sz +successfully over and over, but with the getty babble at the end. Then, +after trying lsz and then going back to sz, every attempt at sending a file +quits with "Got ZCAN". The difference has to be that Kermit always does at +least some minimal encoding of C0/C1 control characters such NUL and DEL and +IAC, and I doubt that Zmodem does. + +http://zssh.sourceforge.net/ says: + + If file transfer is initiated but never completes (ie a line like : + + Bytes Sent: 0/ 513 BPS:0 ETA 00:00 Retry 0: Got ZCAN + + can be seen, but transfer never completes), chances are the pty/tty on one + of the systems are not 8-bit clean. (Linux is 8-bit clean, NetBSD is not). + Using the -e (escape) option of rz should solve this problem. + +It doesn't, at least not with lrz. And yes, the receiving end happens to be +NetBSD. But it looks like the zssh people have been down this road too. + +But with rz and sz, it worked. Once. Twice. Three times. But of course, +with the getty babble at the end. This can be taken care of by doing: + + rz -eq ; cat > foo + +which puts "sz 3.73 1-30-03 finished" and any other messages in foo (but you +have to type ^D to finish the cat). Using this method I was also able to +send an 8K binary file that contained a test pattern of all 256 possible byte +values. Then I tried a 3MB binary executable. All OK. So here we go again: + + . sz txt (zst): OK + . sz bin (zsb): OK + . rz txt (zrt): + . rz bin (zrb): + +Downloading fails about halfway through a fairly large file. I tried an +even bigger file, guaranteed to be 100% ASCII; same thing -- halfway +through: "rz: Persistent CRC or other ERROR". But it worked with a smaller +version of the same file (82K versus 2MB). Tried again with the bigger +version, it failed in exactly the same way at exactly the same spot: byte +number 1048320. But this is just ASCII text so it can't be a transparency +problem. Substituting another plain ASCII file of the same size but totally +different contents, it doesn't fail (2.36MB). Back to the previous file, it +fails again, but in a different spot (832960). So it's not totally +deterministic. + +To round things out, I tried downloading the binary test-pattern file; it's +only 8K. This failed. + + -4, --try-4k go up to 4K blocksize + -B, --bufsize N buffer N bytes (N==auto: buffer whole file) + -e, --escape escape all control characters (Z) + -E, --rename force receiver to rename files it already has + -L, --packetlen N limit subpacket length to N bytes (Z) + -l, --framelen N limit frame length to N bytes (l>=L) (Z) + +Tried again with "sz -L 256 -B 256 -4aeq". Doesn't change anything. + +NOTE: Mac OS X rz 3.73 1-30-03 does not support -e. +NetBSD rz 0.12.20 does support -e. + +Thu 22 Mar 2007: It occurs to me that ttpkt() might still be a problem; +maybe it's the network connection and not the pty that is not transparent +enough. To test this theory I did "stty raw ; stty -a" and then copied all +of the flag values into ttpkt in the BSD44ORPOSIX section: + + . rz txt (zrt): OK (2.36MB file, worked 2 out of 3 times) + . rz bin (zrb): "rz: Persistent CRC or other ERROR" + +A little more fiddling with the flags and I got the 8K binary test pattern +to SEEM to download OK (in the sense that rz gave a 0 return code) but the +file itself was truncated, always at 224. If I changed the test pattern +file to not include any bytes with value 224 (0xe0) or 255 (0xff), the +download worked. So we have a transparency problem somewhere. The debug +log shows that all byte values are being received from the network correctly +so the problem has to occur when we try to feed them to the pty. + +But no amount of twiddling with the termios flags seems to let these +characters pass through. Of course, since they are not in the C0 or C1 +control range, "sz -e" doesn't quote them (which it does by prefixing with +Ctrl-X and then adding 0x40 to the byte value so (e.g.) NUL becomes ^X@. +Note that 255 does not cause problems because it coincides with the IAC +character; the remote Telnet server doubles outbound IACs, and Kermit's +ttinc() undoubles them automatically (as the log shows). + +Trying to send a different file (a C-Kermit binary) shows that 255 is the +real killer; the file is truncated where the first one appears (at about +6K), even though some 224's precede it. Going back to the remote-mode test, +I see the same thing happens with the binary test-pattern file, if I send it +from K95 direct to rz-under-C-Kermit-in-remote-mode. So it has nothing to +do with C-Kermit having a network connection. Yet if I send the same file +direct from K95 to rz, it goes OK and the result is not truncated, so it's +not Zmodem either. The data arrives to C-Kermit intact, so the failure is +definitely in writing it to the rz process through the slave and master ptys. + +BUT if I send the same file from K95 to rz-under-ttpty, that works. What's +the difference? Suppose I just transplant ttpty literally into C-Kermit... +It makes no difference. When receiving the test-pattern, it truncates it +in exactly the same place. + +Well, all this is on Mac OS X. What if I move it to a different platform? +OK, building on Solaris and following the exact same procedure, ttptycmd() +doesn't even use the network connection. I think that's because rzsz on +Solaris is hardwired to use the controlling terminal and can't be +redirected, even in a pty? + +Moved to NetBSD. + + . sz txt (zst): Failed ("Got ZCAN") + . sz bin (zsb): + . rz txt (zrt): OK + . rz bin (zrb): + +Well, this is a big mess. Sending doesn't work (or sometimes it does but +reports that it didn't). Receiving... well, actually it's the same thing; +the file is completely transferred but then the final protocol handshake is +lost. The local C-Kermit returns to its prompt, but rz is still running: + + Retry 0: Got TIMEOUT + Retry 0: TIMEOUT + Retry 0: Got TIMEOUT + Retry 0: TIMEOUT + Retry 0: Got TIMEOUT + Retry 0: TIMEOUT + Retry 0: Got TIMEOUT + Retry 0: TIMEOUT + Retry 0: Got TIMEOUT + Retry 0: TIMEOUT + Retry 0: Got TIMEOUT + +I don't see how that is even possible. Even after I exit from Kermit the +messages keep coming, even though ps doesn't show the rz process anywhere. +Looking at the code, I see a place where end_pty() was still commented out +from the ttpty.c episode, I uncommented it. But still: + + . sz txt (zst): Fails ("Got ZCAN") + . sz bin (zsb): Fails instantly (but with no diagnostic) + . rz txt (zrt): OK + . rz bin (zrb): Fails with tons of "Bad CRC", "Garbage Count exceeded" + +Conclusion for the day: I think this is hopeless. Even if I can get it to +work somewhere, the results depend on the exact Zmodem software, how it uses +stdin/out vs stderr versus getting its own nonredirectable file descriptor, +versus the Zmodem version on the other end and which options are available +on each, versus the pty and select() quirks on each platform, and on and on. +It will be so hard to explain and to set up that nobody would ever use it. +It would be better to just implement Zmodem internally. + +Fri 23 Mar 2007: Went back to the small test program, ttpty.c. Tried +setting both the master and the slave pty to rawmode, even though I have +never seen any other software that did this. I had it receive the binary +test pattern file; it worked. I made a bigger test-pattern file, 3MB, +containing single, double, and triple copies of each byte in byte order and +in random order, this one was accepted too. + +So it would seem that the ckupty.c module is something to avoid after all. +It's full of stuff I don't understand and probably should not undo. So +changing C-Kermit's ttptycmd() to manage its own pty again, using openpty() +(which is not portable), I got it all to work in remote mode: Kermit +text/binary up/down and Zmodem text/binary up/down. But in local mode on +the client side of a Telnet connection... + + zst: OK, but we still get the getty babble at the end that starts sz. + zsb: OK, ditto. This is with the 3MB test-pattern file. + zrt: Not OK -- "Persistent CRC or other ERROR" + zrb: Not OK -- got the cutoff at 224 again "Persistent CRC or other ERROR" + +It's close. But actually this was still with USE_CKUPTY_C defined. When I +undefined it, it was back to being totally broken. Start over. (Check the +new cfmakeraw() code.) + +Tue 27 Mar 2007: Starting over. Back to ttpty.c. Let's verify, VERY +CAREFULLY, that it really does work, using the most stressful of the four +tests: sending the big (3.2768MB) binary test pattern from K95 into rz +through ttpty, logging everything. ttpty definitely receives the big file +smoothly with no errors or hiccups when I have it set to use the master side +of the pty for i/o. The application program (Zmodem in this case) runs on +the slave, and the network and/or control program communicates with the +master. This implies that Zmodem controls the terminal modes of the slave, +and ttpty should be concerned with those of the master. Doing it this way +in ttpty confirms this. + +Fine. But if I tell ttpty to SEND a file with sz, nothing happens. Ditto +with lsz. Select times out waiting for input from the pty. But if I +manually tell K95 to RECEIVE /PROTOCOL:ZMODEM it works OK. Somehow sz's +initial B000000 string is being swallowed somewhere, and it's waiting for +a reply from the receiver. sigh... But "ttpty gkermit -s filename" works +fine. What's the difference? It has nothing to do with stdout vs stderr; +sz is not writing to stderr at all. Is it some timing thing between the +forks? Aha. It's that I change the modes of the pty master in one fork +while sz is already starting in the other fork. + +OK, good, now for the first time we have Kermit and Zmodem both able to +upload and download a large worst-case binary test-pattern file... in +remote mode. Now taking today's lessons and fitting them back into +C-Kermit so I can try it local mode... + +Using G-Kermit as the external protocol, first in remote mode... All good: +text/binary up/down. The "halting problem" is solved by SIGCHLD, which +catches fork termination instantly and lets ttptycmd() know there is no more +pty. Zmodem: + + zst: OK + zsb: OK + zrt: OK + zrb: OK + +That's a first. Next, repeat in local mode, in which C-Kermit is the client +and has made a Telnet connection to another host over a secure (Kerberos V) +connection: + + kst: OK zst: ... + ksb: OK + krt: OK + krb: OK + +It seems we can never end a day on a high note. Somehow I seem to have +broken regular internal Kermit protocol transfers over encrypted connections +-- the en/decryption engine loses sync. But they still work OK over a +clear-text Telnet connection. + +Today's code in ~/80/dulce.tar (27 Mar 2007). + +Added makefile target solaris10g+openssl. Gathered all the standard CFLAGS +for Solaris into cdcdeb.h so they don't have to be included in every single +makefile target for Solaris. On local Solaris 10 host OpenSSL is in +/opt/openssl-0.9.8e/. Tried the new makefile target, works OK. Also made +solaris10+openssl for Sun CC, but couldn't test it because I can't find any +Solaris 10 host that has Sun CC. Built with gcc at another site that has +OpenSSL 0.9.8f-dev, all OK. ckcdeb.h, makefile, 24 Jun 2007. + +It occurs to me that Kermit transfers on secure connections might have been +broken by the changes I made back in February to ttinl() for John Dunlap. +Here, for the first time, we invoke myunrd() to push a byte back into the +input queue, and there is also some funny business with "csave", which +changed, and which an old comment notes that it has to be treated specially +when encrypting. So it could be that the broken Kermit transfer has nothing +to do with the work on external protocols, and that putting back the +previous ttinl() will fix it. But now I can't seem to make a Kerberized +connection from Panix to Panix, even though I can make one from Columbia to +Panix. This means I have to build a Kerberized binary from the current +source code on either Solaris or Mac OS X. Trying Solaris +first... [~/solaris9k5/mk5.sh] This didn't work the first time due to +undefined krb5_init_ets, which is referenced if MIT_CURRENT is not defined +(it should be for Kerberos 5 1.05 and later and we have 1.42 here), tried +again with -DMIT_CURRENT=1... Nope, that one totally blew up in ck_crp.c. +Later, Jeff says krb5_init_ets is a no-op in Kerberos 1.4.x and later, +so I added an #ifdef (NO_KRB5_INIT_ETS) for skipping it; now it builds and +runs OK. ckuath.c, makefile, 9 Jul 2007. + +Meanwhile, using C-Kermit on Mac OS X, which makes the Kerberized connection +just fine, but still has the problem transferring files over it. Packet log +shows: + + s-00-01-^A9 Sz/ @-#Y3~Z! z0___F"U1@A^M + r-00-01-^A9 Y~/ @-#Y3~^>J)0___J"U1@I + s-01-01-^A(!Fx.x)(V^M + r-xx-08- + S-01-08-^A(!Fx.x)(V^M + r-xx-08- + S-01-08-^A(!Fx.x)(V^M + r-xx-16- + +Note that S packet is sent, received, and Ack'd OK. The F packet is sent but +is never Ack'd. Tried this several times and noticed that it's just +receiving that is screwed up, not sending. After ^C'ing out of the +transfer, I can still type commands, and they are executed on the far end, +but the results coming back are gibberish. Mon Jul 9 16:08:22 2007 (come +back to this later... substitute Dev.27 ttinl for current one and see if +the problem goes away, and if so, conditionalize the new code for clear-text +connections). + +Built C-Kermit with Kerberos 5 on Solaris with a version of ckutio.c that +uses the old ttinl() and transferred a file OK over a Kerberized connection. +So now it's just a matter of reconciling the old and new ttinl. The easiest +way to do this is to have new ttinl() chain to old ttinl() if the connection +is encrypted, which is what I did and it works fine. At some point the two +versions of ttinl() should be reconciled. ckutio.c, 12 Aug 2007. + +There was a function, islink(), used in only one place (ckuus6.c) that had +the same name as a commonly used scalar variable, and it was missing a +prototype. Changed its name to isalink() and added the prototype (Unix +only), ckuus6.c, ckufio.c, ckcdeb.h. 12 Aug 2007. + +Revisiting the ASCII and BINARY top-level commands, which are supposed to +be like in other FTP clients, but don't seem to have any effect. I added a +new routine to the FTP module, doftpglobaltype(), that sets the global, +sticky, permanent transfer mode (ASCII or BINARY) (TENEX could be added to +if anybody asks). These commands (now that they work) are different from +SET FTP TYPE { ASCII, BINARY }, which set the *default* transfer mode when +automatic switching fails for a given file. ckuusr.c, ckcftp.c, 12 Aug 2007. + (notify: Matt ) + +Even though the code hasn't changed, suddenly we're getting: + + "ckuusx.c", line 5682: warning: implicit function declaration: tgetent + "ckuusx.c", line 6183: warning: implicit function declaration: tgetstr + "ckuusx.c", line 6262: warning: implicit function declaration: tputs + "ckuusx.c", line 6266: warning: implicit function declaration: tgoto + +in ckuusx.c on Solaris 9. is still in /usr/include, dated 2002. +A quick search shows the missing functions are hiding in , which +until now was included only in Linux. Added a USE_TERM_H clause. No, that +doesn't help, the prototypes are not selected at compile time; there are +#ifdefs in that file that skip over these prototypes. I had to put them in +the code under #ifdef BUG999..#endif (I could have used a longer name like +#ifdef ADD_PROTOTYPES_FOR_CURSES_FUNCTIONS, but that would not be portable). +ckuusx.c, 12 Aug 2007. + +Also: + + "ckuusx.c", line 9232: warning: implicit function declaration: creat + +This is called in the IKSD database code, used for getting a lockfile. +creat() is a Unixism in code that is supposed to be portable. But IKSD only +runs on Unix and Windows, so I assume the Windows C library has a creat() +function. Anyway, suddenly the Solaris header files seem to have blocked +whatever path previously existed to the creat() prototype (which is in +), so I added an #include in the appropriate spot. ckuusx.c, +12 Aug 2007. + +Kermit functions for converting the number base -- \fradix(), \fhexify(), +\unfhexify() -- did not work with big numbers; ckradix() was missed in the +CK_OFF_T conversion. Fixed in ckclib.c, 12 Aug 2007. + +Updated the help text for ASCII, BINARY, and SET FTP TYPE to clarify the +semantics. ckuus2.c, ckcftp.c, 12 Aug 2007. + +Error messages were printed upon failure to open any of the four log file, +even with SET QUIET ON. Fixed in ckuus4.c, 12 Aug 2007. + +Built OK on NetBSD 1.3_RC3. Tried to build secure version but the libraries +had disappeared. 13 Aug 2007. + +Built OK on Mac OS X 10.4.9. Tried the secure version, macosx10.4+krb5+ssl. +Here we get the usual pile of "pointer targets in passing argument 1 of +(function name) differ in signedness", regarding security functions, but it +built OK. 13 Aug 2007. + +Reconciling the two ttinl's... On encrypted connections myread() returns +encrypted bytes; ttinl() has to decrypt them; it wasn't doing this in the +lookahead section so I fixed it. The new code works on both encrypted and +clear-text connections. I removed the chaining to oldttinl(), and +oldttinl() itself. ckutio.c, 13 Aug 2007. + + (Wouldn't it make more sense and be more efficient and less confusing + for myfillbuf() to do the decrypting?) + +When C-Kermit uses Zmodem as an external protocol, it doesn't seem to scan +files before sending them to set text or binary mode appropriately. It's +that external protocols bypass Kermit's whole "get next file" mechanism; the +(possibly wild) filespec is simply passed to the external protocol program. +Changing this would be a very big deal. But if only one file is being sent +(the filespec is not wild) it's easy enough to check. I added this to the +external protocols section of the protocol module. It can be overridden in +any of the regular ways (/TEXT or /BINARY switch on SEND command, SET +PATTERNS OFF, SET TRANSFER MODE MANUAL, etc). ckcpro.w, 13 Aug 2007. + +[FTP SEND /RECURSIVE] +Peter Crowley reported a problem with FTP recursive uploads getting the +directory tree wrong when the previous pathname was a left substring of the +new pathname (e.g. foo/bar/ and foo/bar2/). The logic did not handle this +case and created the bar2 directory as a subdirectory of bar, rather than as +a parallel directory. Fixed in syncdir() and tested with various edge cases. +ckcftp.c 14 Aug 2007. + + notify + +Added CD messages to FTP BRIEF display to track the ups and downs of +recursive uploads. ckcftp.c, 14 Aug 2007. + +The OUTPUT command gave a misleading error message ("Connection to xxx not +open") when used on a serial port that was, indeed, open but was not +presenting the Carrier signal, when CARRIER-WATCH was not OFF. Added a new +message for this, and some others. ckuus5.c, 14 Aug 2007. + +Sending from the command line, e.g. kermit -s foo, did not give an +informative error message if the file could not be found or opened. Fixed +in ckuusy.c, 14 Aug 2007. + +OK, back to ttptycmd.... It seems that back on March 27th, I got everything +working but I thought that there was still something wrong with it because +an unrelated problem so I put it aside. The version of ttpty.c from that +date worked OK, and it looks like I updated ckutio.c from it, but that +version of ckutio.c was put aside. Since then I have been working on the +ckutio.c version that was NOT put aside and so now I have to reconcile the +two: + + ~/80/ttypty/20070327/ckutio.c + ~/80/ckutio.c + +As a first cut I did this simply by replacing the contents of the #ifdef +CK_REDIR section of the latter with that of the former. Of course in +Solaris this comes up with openpty() implicitly declared at compile time and +unresolved at link time. So the first task is to get HAVE_OPENPTY defined +for platforms that have it and have the others use the ttruncmd(). For +starters I put an #ifdef block in ckcdeb.h that defines HAVE_OPENPTY for +Linux, FreeBSD, NetBSD, OpenBSD, and Mac OS X. Ones that don't have +openpty() include AIX, HP-UX, and Solaris. Others like SCO I don't know but +I doubt it. The real solution is to get the ckupty.c module to work but one +thing at a time... This version is supposed work with secure builds on the +openpty() platforms, and on the others like Solaris, if an external protocol +is attempted on a secure (encrypted) connection, an error message is +printed and the command fails. ckutio.c, 14 Aug 2007. + +How to test? Apparently I did all my testing on Panix before, and that's +where all my Zmodem builds are, but now when I build a Kerberized version +(which works if I do it on the right pool host), it won't make a local +connection, and there is no other place I can connect to that has a +Kerberized Telnet server. I can, however, connect to Panix from here, using +the same code, but on Mac OS X... + +Slight detour: Got access to AIX again (5.3.0.0). Picky compiler, some +things needed fixing.... Also it says "1506-507 (W) No licenses available. +Contact your program supplier to add additional users. Compilation will +proceed shortly" and of course it goes kind of slow. For some reason, I +can't do streaming transfers into AIX over a local network (to its SSH +server), but windowed transfers are OK. Anyway, noting that we've been +using the same basic makefile target since AIX 4.2, changing nothing but the +version herald, I made a new target, simply "aix", that picks up the AIX +version automatically and sets the herald from it. Ditto for aix+openssl, +but on this host requires setting SSLINC and SSLLIB to /opt/ssl/include and +/opt/ssl/lib. Also the make program here was extremely sensitive to spacing +so I had to make some minor edits to get the link step to work for the SSL +version. ckuusy.c, makefile, 14-15 Aug 2007. + +Got rid of the special Panix secure NetBSD target, replaced it with a +regular one, which is invoked in the normal way by defining K5INC and K5LIB +to point to to where the stuff is hidden. Cleaned up and modernized the +comments in the makefile a bit. makefile 15 Aug 2007. + +Changed some data types and added some casts to ckctel.c to do away with +tons of "pointer targets in passing argument 1 of 'xxx' differ in signedness" +warnings. 15 Aug 2007. + +Set up Mac OS X as the testbed for ttptycmd(), with Panix as the remote +partner over a Kerberos 5 connection. The first test is to send a 300K +text file with gkermit as the external protocol. It worked fine, and the +debug log showed all the right components were active (namely encryption and +ttptycmd) [kermit/zmodem send/receive text/binary]: + + Kermit Zmodem + kst OK zst OK + ksb OK zsb OK + krt OK zrt OK + krb OK zrb Failed "rz: Persistent CRC or other ERROR" + +We've seen this before. The problem is 0xff, Telnet IAC, as I proved to +myself by constructing a 3MB file that contained every byte but 0xff in every +mixture and order and transferring it successfully over the same connection. +Presumably the Telnet server is doubling IACs, whereas of course rz is not +undoubling, thus the CRC error. This is progress. 15 Aug 2007. + +Log shows that indeed every IAC in the source file arrives doubled. Adding +code to remove the first IAC of every adjacent pair, a small test file with +different-length runs of IACs transfers OK. The 3MB all.bin file does not. + +Starting over... I can receive a big text file with Zmodem OK. The 3.2MB +binary test pattern that contains no IACs failed after 1.8MB, but the part +that it transferred was OK. A second try, almost the whole thing arrived, +it stopped just 584 bytes short of the end. Could be that file size is a +separate problem. Making a new copy exactly 1MB long... Well, that's +interesting, this one too stopped just short of the end. And again, the +same thing. When connecting back to the host, the last Zmodem packet can +be seen on the screen; it's as if the local Zmodem exited before reading +the last packet... But OK, if I change the options on the remote sz +sender to use small blocks, etc, then it works. + +Now, changing from the 1MB no-IAC-binary test pattern, to the 1MB all-values +test pattern, we fail after 81K. But the part that was transferred is +correct. Second try, same thing, but 57K. Third: 40K. Each time, upon +connecting back, the session is completely dead. + +IF I HAVE TO undouble IACs for incoming files, don't I have to double them +going out? To send a block to net we just call ttol(), but ttol() doesn't +do any doubling (because Kermit protocol always quotes 0xff). To see what +happens, I changed the ttol() call to ttoc() in a loop that doubles IACs. I +tested this by sending the full 3.2MB test pattern, which worked fine. + +For receiving, it's slow but it works OK with files that don't contain IACs +(my concern was that IACs might appear in outbound files or in Zmodem +protocol messages). It receives the 1MB no-IAC test pattern, so there are +no problems with protocol or timing. But the full test pattern always gets +cut off, but at different points, as before, with the remote session dead. +Changing the Zmodem receiver from rz to lrz on the local end (since the +sender on the remote end is lsz) does not change the behavior. + +Anyway, I went back and replaced the byte loop with something more +efficient, and it goes about 20 times faster. But this doesn't help either, +it only makes it fail faster. But aha, what if a doubled IAC is broken +across successive pty reads -- we have to make the "previous character" +memory persistent. Well, that was a good insight, but it still didn't fix +it. The log shows the IAC handling code is working fine. + +What does sz say? Capturing its stderr to a file... "Retry 1: Got ZCAN". +Next time: "Retry 1: Got TIMEOUT". Next time: Got ZCAN. + +Trying different Zmodem options... apparently I don't need to use short +blocks. But I do need to use -e, probably because of Telnet NVT treatment +of carriage return; without -e, there is a "persistent CRC error". -O +disables timeouts, but this makes no difference. + +OK, we still have two Big Problems: + + 1. When a long file has no IACs, the final < 1K of the file is not received. + 2. When a long file has IACs, the transfer generally stops very early. + +Problem 1: the transfer consistently fails less than 1K from the end of the +file. Upon CONNECT back to the host, a big Zmodem packet is sitting there +waiting to be read, which means ttptycmd()'s copy of rz is terminating +early. Can we catch it in the debug log? Doing this takes forever and +writes a GB to the disk... And then the problem doesn't happen. Also, I +can receive a HUGE text file almost instantly with no errors at all. + +Switching to lrz on the receiving end, now I see the error messages, about +300 lines like this: + + Retry 0: Garbage count exceeded + Bytes received: 872352/1000000 BPS:85464 ETA 00:01 Retry 0: Bad CRC + Bytes received: 892448/1000000 BPS:86690 ETA 00:01 Retry 0: Bad CRC + Retry 0: Garbage count exceeded + Retry 0: Garbage count exceeded + Retry 0: Garbage count exceeded + Retry 0: Garbage count exceeded + Retry 0: Garbage count exceeded + Retry 0: Garbage count exceeded + Retry 0: Got ERROR + Bytes received: 898336/1000000 BPS:84293 ETA 00:01 Retry 0: Bad CRC + Retry 0: Garbage count exceeded + Retry 0: Garbage count exceeded + Bytes received: 900384/1000000 BPS:83751 ETA 00:01 Bad escape sequence + 2fRe + try 0: Bad data subpacket + Bytes received: 941472/1000000 BPS:86191 ETA 00:00 Retry 0: Bad CRC + Retry 0: Garbage count exceeded + +Even when it succeeds, it gets these. But if I receive a text file, no +matter how big, no errors or retries or timeouts at all. So it appears that +there is only one problem: a big-time lack of transparency regarding 8-bit +and/or control characters. The odd thing is, it's not that the characters +can't get through -- they all can -- but they seem to cause transitory +blockages. 16 Aug 2007. + +Cleaned up the remaining pointer signedness warnings in ckutio.c, but this +was a mistake, it broke Kerberos connections completely. Undid the changes. +ckutio.c, 17 Aug 2007. + +Changed all return() in the fork()==0 section of ttptycmd() to exit(). +ckutio.c, 17 Aug 2007. + +Tried explicitly setting the slave pty to rawmode. Makes no difference. +Tried using the Mac OS X (curses) raw() function, and also system("stty +raw"); still no difference. Tried doing all of these in different +combinations and orders. I found one combination that cuts the errors about +in half, and the transfer of the no-IAC test pattern almost always succeeds +(but it's slow). Anyway, it doesn't help much with the test pattern that +contains IACs. Well, the code is more solid than it was before but +functionally we have not advanced much if we can't download a binary file +with Zmodem! On the other hand, we can upload them, and we can transfer +text files in both directions, which is an improvement over the previous +situation, in which the entire session would hang due to loss of +synchronization of the encryption stream. + +Tried adding -funsigned-char to CFLAGS of Mac OS X target. It does not +make the "signedness" warnings go away and it doesn't change the runtime +symptoms. + +I tried a simpler version of pty_make_raw(), the one from Serg Iakovlev, but +it was a total failure. That's encouraging though, because it indicates +that pty_make_raw() is the right place to be working. + +Then I made pty_make_raw() set or unset every single terminal flag +explicitly. This made no difference, but didn't hurt anything either. + +Then I made pty_make_raw() explicitly set all the c_cc[] characters to 0 +(but left c_cc[VMIN] as 1). This made no difference either. + +I checked pty_make_raw() against ttpkt() and the only difference I found in +the terminal flags is that ttpkt() sets IGNPAR thinking it means "ignore +parity errors" when really it means "discard any character that has a parity +error" (at least according to Iakovlev) -- exactly the opposite. But I +tried it both ways, no difference. 17 Aug 2007. + +I noticed that even Zmodem text receives can fail. They don't get any +errors, they just get cut off shortly before the end. (But usually they +succeed, and fast too, like 500K cps). + +What if I don't call pty_make_raw() at all on the slave pty? + +zrt: EESSSSSSSS: 80% good (E = stopped just before end but no other errors) + +zrb no-IAC test pattern, short blocks: + 1. S/5 (success with 5 screens of errors. + 2. S/7 + 3. S/7 + 4. S/6 + 5. E/7 (failed just before end) + 6. S/7 + 7. S/6 + 8. S/6 + 9. S/6 +10. S/4 + +So, lots of errors, but it recovered 90% of the time. +Next, same thing, but without requesting short blocks: + + 1. E/5 + 2. S/5 + 3. E/4 + 4. S/5 + 5. S/5 + 6. S/5 + 7. X/0 (hard failure right away: "Got ZCAN" + 8. S/5 + 9. S/5 +10. S/5 + +So it doesn't look like short blocks make that much difference. Now what if +I turn off prefixing? Bad CRC, fails immediately every time. Putting back +pty_make_raw(slave), it still fails hard. + +Tried a new strategy with pty_make_raw(): rather than modify existing flags, +I set all flags to 0, and then turn on only those few that we need like CS8. +Now we get only 2.5 screens of errors instead 4-7 and the transfer rate is +higher for binary files (all of the previous ones were under 100K CPS, while +for text files it was 400-500K CPS): + + 1. S/2 195669 CPS + 2. S/2 194720 + 3. E/3 + 4. S/2 192550 + 5. S/3 192325 + 6. S/3 145066 + 7. S/2 200689 + 8. S/3 188948 + 9. S/2 209461 +10. S/3 181991 + +I noticed that there was no TIOCSTTY ioctl in the pty/fork setup sequence, +which is recommended somewhere, so I tried that and it was a disaster; the +entire session hung. I took it back out. 18 Aug 2007. + +Tried some transfers over a clear-text (not encrypted) connection with the +same results: smooth, fast transfer of a big text file (400K cps); rocky but +successful transfer of the no-IAC binary pattern file (135K cps). Switching +back to ttruncmd(), the same binary file is received at 1.5M cps, and the +no-IAC binary file totally fails after too many "Bad CRC"s; and we already +know that any file that contains IACs will fail. One might say that +ttptycmd() is better in every respect than ttruncmd() except in speed +(when it works). + +Let's see if ttyptycmd still works in remote mode (to local K95): + . sz / text works, but slowly. + . lsz / text works but some weird errors are reported. + . lsz / binary / no IAC doesn't work at all (CRC-32 mismatch for a header; + Unexpected control character ignored: 13, etc). + . sz / binary / no IAC works OK but slow. + . sz / binary / full test pattern with IAC works OK but slow. + . Sending text into rz fails completely. + +What about ttruncmd() in remote mode? + . send /text works, fast. + . send /binary works, fast. + . receive /text works, not so fast but not bad. + . receive /binary works, not so fast but not bad. + +So we use ttruncmd() for remote mode, and we use it for local mode +serial-port and modem connections, and we use ttptycmd() on network +connections because (a) they might be encrypted, and (b) even if they are +not, they use some protocol that we have to handle, e.g. Telnet, Rlogin. +19 Aug 2007. + +Discovered that Sending binary files no longer works. Text is OK, binary +transfers don't even start. This happens on both encrypted and clear-text +connections. ttptycmd() is being used in both cases. But oddly enough, +receiving binary still works as before. What did I break, and when? +Oh, it was just the script, when I changed it from using sz to lsz. Putting +it back to sz makes it work, even with the full 3.2MB binary pattern with +IACs. + +I backed off the changes I made to ckctel.c to suppress some warnings, in +view of the fact that similar changes to ckutio.c broke things so badly. +19 Aug 2007. + +If sz is not given the -e flag, it sends control characters bare, except ^P, +^Q, ^S, and ^X. ^X is the control prefix, so ^A is sent ^X followed by A. +With -e, all C0 control chars are prefixed, but with ^X, which is, of +course, a control character. Interestingly, the C1 analogs of ^P, ^Q, ^S +(but not ^X and, unfortunately, not IAC) are also prefixed. -e makes no +difference for 8-bit characters. + +If we have a Telnet connection and the server is in ASCII (NVT) mode, CR is +always followed by LF or NUL. Well, it seems the server is putting us +(Kermit) in binary mode in this case, but staying in ASCII mode itself. +Added code to handle NVT byte stuffing and unstuffing in each direction +independently, according to the TRANSMIT_BINARY state in that direction. I +made a file containing just the bytes 0-31 and 127 and 128-159 and 255 (66 +bytes all together) and sending it from the host to C-Kermit, the local log +shows that every control character was received correctly and all TELNET +conversions were done right -- NUL removed after CR (and only after CR); IAC +removed after IAC (and only after an IAC meant as a quote). For the first +time, I can receive the 1MB all-values test pattern, but there are still +tons of (correctable) CRC errors, so the transfer rate is really awful, like +about 5% of what we get with a text file (25Kcps instead of 500). + +Further experimentation shows that the fundamental transparency problem is +fixed; we can receive short files (say, 1K or less) containing absolutely +any byte values in any combination with no errors at all. But once the file +size reaches (say) 10K, we get CRC errors, like one every 2 or 3K of data. +These are not deterministic. In successive transfers of the same file, they +come in different spots. It's tempting to blame pty buffer overruns, but +then text files would show the same behavior. When a binary file size +exceeds, say, 1MB, the chances of successful completion go way down, +independent of whether my external protocol is rz or lrz. I like lrz better +because the error reports come out on the screen as the transfer is going +on. Trying to download a real-world binary file -- a 2.2MB C-Kermit +executable -- I get 4500 error messages but the transfer eventually succeeds, +with an effective throughput of 21Kcps. + +Actually it turns out that "sz -a somebigtextfile" (2.2MB) also gets a lot +of CRC errors. The -e flag (escape all control characters) makes the same +big text file transfer with few or no errors. It's not sure-fire. +Sometimes no errors, sometimes one or two, and sometimes a fatal error that +kills the transfer. + +With binary files... a 32K binary file seems to make it every time. 40K +fails about 50% of the time. 48K fails 60% and every time it fails, it has +created a partial file of exactly 32K (32768 bytes). 96K fails 9 out of 10 +times, when it fails, the partial file is always 0 bytes, or 32768, or +65536, but that just means that rz's file output buffer is 32K. + +Why, then, do binary files cause trouble if it is not a solid transparency +problem? If a certain file can get through once, why can't it get through +every time? When a character arrives at the pty, the pty driver probably +takes a different path through its code, checking the terminal flags that +would affect that character. I tried making Kermit's network read buffers +very small but, surprisingly, this made things worse. I also tried making +them very much bigger, which didn't help either. 24K still seems to be the +right size. + +So, is it that some characters take longer to process than others? So long +that data is lost due to lack of flow control between TCP and the pty? One +way to test this theory is to slow Zmodem down. I tried "-l 32" which, +according to the man page, tells sz to "wait for the receiver to acknowledge +correct data every N (32 <= N <= 1024) characters. This may be used to +avoid network over-run when XOFF flow control is lacking." Makes no +difference. I also tried the -w (Window) switch, ditto. In fact there are +all sorts of options to set the "window size", "packet length", "block +size", and "frame length", but with no explanation of what these mean or how +they are related. If I crank everything down to minimum value: + + lsz q -L 32 -l 32 -w 1 + +I get 50% success with the 96K file instead of 10%. Adding -e, oddly +enough, made it worse. I also tried setting the environment variable +ZNULLS to different numbers like 512, no help there either. + +I tried making the read-from-net-write-to-pty buffer small (1K) but leaving +the pty-to-net one big. This improves chances of success, but it's +intolerably slow (3Kcps when the connection is capable of 500K). + +I also changed the write-to-pty operation from a single write() call of +possibly many K characters to a byte loop, one write() per byte. Same +result: success (but still about 300 recoverable errors), throughput 3Kcps. +20 Aug 2007. + +With ttptycmd() configured to write to the pty in a byte loop, it is +possible to delay each write. Adding a 10msec delay per character results +in a transfer that runs at about 20 cps and (for the 96K test file) would +take about 80 minutes to complete. And yet it still gets just as many +errors. So it's not a matter of timing either. The errors come, on +average, every file 388 bytes, but not at regular intervals. + +I tried the TIOCREMOTE ioctl on the pty master, as discussed somewhat +obliquely in the Mac OS X "man pty" page; "This mode causes input to the +pseudo terminal to be flow controlled and not input edited (regardless of +the terminal mode)" -- sounds like just the ticket but it made no +difference. Actually, looking at a man page on another OS (Solaris), it +says this is only for lines of text, EOLs are supplied, so that would mess +up the protocol. So remember: don't use this. + +Tried without O_NDELAY; the behavior was the same but the speed was much +slower. + +Tried switching back to the ckupty.c routines on Mac OS X and found that it +works now the same as with openpty(), except that I seem to get more getty +babble at the end. But this means I can run some tests on Solaris. I moved +the entire test environment from Mac OS X 10.4.9 to Solaris 9. But it +doesn't work at all. + +Trying to figure out the ckupty.c modules again. + . do_pty() calls pty_getpty() which returns in arg1 the fd of the pty master. + . Then it creates a pipe as a way to tell when the child dies + . Then it creates a fork: + - The parent does a blocking read from the pipe + - The child calls getptyslave() to get the pty slave + and writes one byte to the pipe + and then execs the command it's supposed to run +Note that the file descriptor of the slave is known only to the lower fork. +Therefore the lower fork is the one that has to set all the tty modes, etc. +I took care of all that but the ckupty.c method doesn't work at all on +Solaris. But it works "fine" on Mac OS X (the 32K all-bytes test file +transfers instantly with no errors, but the 96K one errors out). + +The problem on Solaris is that pty_make_raw() fails on the masterfd (but not +on the slavefd) with errno 25 "ioctl inappropriate for device". It doesn't +matter whether I do it in ckupty.c or ckutio.c. I found a web page on +kde.org that says Solaris does not allow tcget/setattr() on a pty master. +But the Sun "knowledge base" is not open to the public. Well, presumably +changes made to the slave are reflected in the master (comments in Solaris +telnetd seem to confirm this...) Let's come back to Solaris later. + +Moving to a Linux with lrzsz installed... Built a Kerberos 5 version with +USE_CKUPTY_C. Like on Mac OS X, it transfers short files OK and chokes on +longer ones. Switched to openpty(), it behaves the same. So the problems +on Mac OS X are evidently not OS-specific, which is good I guess, since that +means finding the way around them will apply to more than one platform. +21 Aug 2007. + +Look into TIOCSCTTY again. On System V based OS's, opening a pty acquires a +controlling terminal automatically. On BSD-based OS's, no; you have to use +the TIOCSCTTY on the slave file descriptor to give it one. I'm not sure why +a controlling terminal would be needed, except that without one, the virtual +device "/dev/tty" does not exist for the process that runs on the pty, and +maybe the application that runs there (e.g. rzsz) checks for it. On the +downside, having a controlling terminal opens the process up to terminal +interrupts like SIGINT and SIGQUIT. Until now I have not been using this +ioctl(). Results (in Linux): + + With TIOCSCTTY: 96K all-bytes test: 11 screens of errors, then success + Without TIOCSCTTY: exactly the same. + +Tried the same thing with TIOCNOTTY instead of TIOCSCTTY, with exactly the +same results (no effect whatsoever). + +There has to be a way to make this work, because Zmodem works through +telnetd, which basically the same thing as ttptycmd(): a relay between the +network and a pty. ttptycmd() is like telnetd backwards. Modern telnetds +are not much help; they don't access ptys or the network directly, they go +through "mux" devices so I can't see what they're doing to get transparency +and flow control. An old BSD telnetd uses packet mode but that would be a +big deal... + +I tried ignoring various signals like SIGTTOU and SITSTP, since some Telnet +clients do this. No effect, no difference. Anyway, in Linux the transfers +almost always finish OK despite the many errors. There is just some trick +I'm missing to make the pty accept a stream of arbitrary bytes without +hiccuping. + +What about Solaris, which uses ckupty.c? In streams-based OS's, where line +disciplines and whatnot are pushed on top of the pty, it looks like the pty +module saves the file descriptor of the "bare" slave pty (as 'spty') before +pushing things onto it, and then later uses spty rather than the regular +slave pty file descriptor when getting/setting terminal modes. I'm not sure +what this is all about but it's definitely SysVish... It happens if +STREAMSPTY is defined, but I noticed that STREAMSPTY is never defined +anywhere. I tried defining it so we take an entirely different path through +the code. It made absolutely no difference. + +Then I noticed that HAVE_STREAMS is not defined for Solaris either. Tried +defining it, but the session didn't work at all, no i/o. Removing the +HAVE_STREAMS definition but keeping the STREAMSPTY defined, I rebuilt and +tried "set host /connect /pty emacs". I got an EMACS screen but could not +type anything into it, which means that STREAMSPTY should not be defined +either. Removed the definition and "set host /pty" works again. So what's +the problem with ttptycmd()? + +In fact, ttptycmd() works on Solaris with Kermit as the external protocol, +but not with Zmodem, not even with text files. So again, there is no +fundamental problem with the code or the logic, it's Just A Matter Of +Transparency to control and/or 8-bit characters -- some trick I don't know +about. + +Looking at the Solaris debug log... I see that ckupty.c is calling +init_termbuf() to set the tty modes of the master, not the slave, and +set_termbuf() to set them, but you can't do that in Solaris, error 25. This +is in getptyslave(). Shouldn't getptyslave() be setting the tty modes of +the slave, not the master? I changed it to do this, but like all other +changes, it made no difference. I checked to make sure that after the change, +"set host /pty /connect emacs" still worked and it did. + +And then what... I had some code to redirect stderr in ckupty.c that was +not being executing due to a typo. When I fixed the typo, poof, Zmodem +binary transfers started working, or working as well as they work in Linux +and Mac OS X. It turns out that if I don't redirect stderr, sz and rz +just don't work. But lsz and lrz do. But if I do redirect it, I don't see +the progress messages from lsz/lrz. 22 Aug 2007. + +Built on HP-UX 11i v3 (B.11.31 U ia64) with optimizing compiler, got tons of +picky warnings, but it finished and linked and runs OK. Many of the +warnings were like this: + + "ckucns.c", line 1606: warning #2068-D: integer conversion resulted in a + change of sign: tnopt[0] = (CHAR) IAC; + +IAC is defined as 255 in ckctel.h. If I define it as 0xff, I don't get the +warnings. I changed the definitions of all the Telnet commands to be in hex +notation rather than decimal. If cuts way down on the HP-UX warnings and +doesn't seem to cause problems elsewhere. ckctel.h, 23 Aug 2007. + +Now it looks like Solaris is working but then it hangs at the end. It +appears as if the ckupty.c module is blocking SIGCHLD. Debug log shows that +when the transfer is complete, we received IAC DM (Telnet Data Mark) after +sz's last gasp and before the shell prompt is printed. But calling +tn_doop() in this case is a mistake because we are reading the number of +bytes that we know are available in a counted loop, but tn_doop() would +consume an unknown number of bytes and we would never know when to exit the +loop. Anyway, C-Kermit doesn't do anything with DM. Skipping over +tn_doop() (and not writing out the Telnet command bytes) fixes the hanging +condition at the end, even though SIGCHLD is never raised. ckutio.c, +23 Aug 2007. + +Some tests, Solaris to NetBSD over K5. +zst sends ascii.txt, a 2.36MB ascii text file (Kcps / Errors). +zrt receives the same file: + + zst 587/0 526/0 542/0 434/0 423/0 + zrt 827/0 800/0 847/0 FAIL 610/0 + +So text is good. Binary not so good. Here we transfer the 1MB all-bytes +pattern file. zrb receives it successfully, but with 1248 errors, at only +15Kcps. Sending the same file out always fails: + + Begin 20070823 16:32:07: SEND BINARY all2.bin [sz] + Sending: all2.bin + Bytes Sent: 5600/1000000 BPS:12446 ETA 01:19 FAILURE + End 20070823 16:32:13 + Elapsed time: 6.617992999999842 + cps = 151103.2121067556 + lsz: caught signal 1; exiting + +Decided to move to Linux but found that something is screwed up in Linux +C-Kermit with tilde expansion: + + send ~/testfiles/all.bin + +doesn't expand at all (but it did yesterday!). The problem was in the +ancient, ancient realuid/setuid handling code; real_uid() no longer works in +Linux. I worked around this in whoami() by setting ruid to getuid() if +real_uid() returned a negative number. Maybe dangerous, worry about it +later. ckufio.c, 23 Aug 2007. + +ANYWAY... after fixing that, I tested zsb on Linux, and it's broken there +too, using openpty(), so it's nothing to do with ckupty.c. After sending +the first Zmodem data packet, it just hangs, nothing comes back. In text +mode it gets farther, but then the same thing happens. Captured stderr from +rz on the far end: + + Bytes received: 608/1000000 BPS:21137 ETA 00:47 Retry 0: Bad CRC + Bytes received: 864/1000000 BPS:23540 ETA 00:42 Retry 0: Bad CRC + Bytes received: 1120/1000000 BPS:25003 ETA 00:39 Retry 0: Bad CRC + Bytes received: 5696/1000000 BPS:56988 ETA 00:17 Retry 0: Bad CRC + Bytes received: 9120/1000000 BPS:62227 ETA 00:15 Retry 0: Bad CRC + Bytes received: 9376/1000000 BPS:60766 ETA 00:16 Retry 0: Bad CRC + Bytes received: 9632/1000000 BPS:60361 ETA 00:16 Retry 0: Got TIMEOUT + Retry 0: Sender Canceled + Retry 0: Got ZCAN + +The local sz, however, doesn't give any error message. ZCAN means: "other +end canceled session by sending 5 ^X's" (or user typed them). What actually +happens is that ttptycmd()'s select() times out waiting for something from +the Zmodem partner and ttptycmd() itself kills the sz fork with SIGHUP. +When lsz receives SIGHUP it sends the ZCAN. So the real problem is that +after some point we're not receiving anything. + +I changed the timeout from 4 seconds to 30 seconds and now I see it just +stops for long periods of time and then resumes. The lrz log on the +receiving end shows tons of timouts, CRC errors, and other errors. The +local log shows that lsz wound up sending ZCAN (2 x (10 x ^H, 10 x ^X)). + +Moving on to another problem... Turns out Ctrl-C (SIGINT) is working right +after all. Since I'm using my test scripts like kerbang scripts, Ctrl-C +exits through trap(), as it should, closing the connection and cleaning up. +If I start Kermit and tell it to TAKE the script, then Ctrl-C brings me back +to the prompt with the connection still open (as it should). However, until +now I haven't done anything about the fork or the ptys. Added code to +trap() to kill the fork and close the master pty. ckuusx.c, 24 Aug 2007. + +Added code to try to break the deadlock. If select() times out, but we have +stuff to write either to the pty or the net, try to do it anyway, even +though select() did not say we could. But this doesn't help because when +select() times out we don't have anything to write. The problem is that +after receiving that last packet from the remote rz, the local lsz doesn't +seem to do anything, as if the lower fork wasn't running (and to confirm +this hypothesis, sometimes I noticed that when I Ctrl-C'd out of this, the +transfer would take off again). + +Backing up and testing with gkermit rather than zmodem: + + kst ripple.txt [824K] OK + kst ascii.txt [1359K] OK + krt ripple.txt -- FAILED + +It seems that we can't handle streaming. If I set up krt to disable +streaming on receipt, it works OK. + + krt ripple.txt [824K] OK + krb all2.bin [1000K] OK + +So here we have no trouble sending but big trouble receiving unless we +disable streaming. Whereas with Zmodem we have trouble receiving. + +But this wasn't happening before, what changed? Using C-Kermit on the far +end to receive the file with debug log on, I see that it is sending 4K data +packet after 4K data packet, with the local gkermit silent, as expected. +About midway through the transfer, the local Kermit sends an error packet +"Transmission error on reliable link". Looking at G-Kermit's debug log... +It receives the first five 4K data packets OK, but gets a CRC error on the +fifth one, and sends the Error packet. So it has received a stream of +20-some thousand bytes OK and then messes up. That number sounds a lot like +ttptycmd()'s buffer size. I changed the buffer sizes to be different: + + Read from pty and write to net: 4K + Read from net and write to pty: 1K + +This time it received the first 4K packet and failed on the second one. +Then I increased the buffers to 98K each, expecting to receive lots more +packets successfully but it bombed out on the 5th one. But that's good, it +confirms there's no logic error in the buffer management. Just to make +sure, though, let's set the buffer size smaller than the packet size and +disable streaming. In this case we get 4 good data packets and a CRC error +on the 5th one and so we request retransmission, and the next 8 times it +arrives it gets a different CRC error, but the 9th copy is OK. Then the +next packet comes and it gets a CRC error every time. And this is nothing +but plain ASCII text. + +Switching to remote mode: + + REMOTE=1 kk kst + +(after tricking myself because it was using ttruncmd() for this...) I see +that nothing works at all. What did I break? 24 Aug 2007. + +Fixed ttptycmd() to restore console modes after a remote-mode transfer. +ckutio.c, 25 Aug 2007. + +Noticed that error codes like ESRCH are not available in all modules. +That's because of some complicated in #ifdefs in ckcdeb.h that wind up not +always #including . But I notice that ckutio.c includes it +unconditionally with no ill effects, and so does ckvfio.c. Does any version +of Unix at all not have ? Added a catch-all clause to ckcdeb.h to +#include (in UNIX only) if, after the other clauses, ESRCH was +still not defined. ckcdeb.h, 25 Aug 2007. + +Now back to debugging ttptycmd()... Remote-mode transfers with ttptycmd() +were broken in two places, maybe as long as 2 weeks ago (this would have +affected non-network transfers too, which I can't test any more). +The logic was missing in a couple places for the non-network and/or +non-Telnet and/or non-encrypting connections (if statements with no else +parts). Fixed in ckutio.c, 25 Aug 2007. + +Testing remote mode: + + kst OK zst OK + ksb OK zsb OK + krt OK zrt OK + krb OK zrb OK + +Functionally it all works but there are hitches with Zmodem as always. +When sending to K95: + + . If I send with lsz, there are hundreds of "Subpacket too long" errors, + and the transfer is very slow, but it succeeds. + + . If I send with the 1994 Omen version of sz, transmission is instantaneous + and without errors, but then it hangs at the end. + + . If I bypass C-Kermit and send direct from lsz or sz, both work fine. + +So clearly the ptys are getting in the way. The hanging at the end would be +caused by the sz process closing before its last output reached the master +pty. It would need to do some form of flushing and/or pausing at the end +but there's nothing I can do about that; these programs were not designed to +be used in this way. Anyway, it only seems to happen with files longer than +100K. + +For local mode, testing in Solaris over our Kerberos 5 connection again: + + gkermit lrzsz + kst OK zst FAIL + ksb OK zsb FAIL + krt OK zrt OK but with errors + krb OK zrb FAIL + +If I use Omen rzsz as the external protocol (e.g. with zst), it blocks +redirection and it sends the file to my terminal, rather than over the +connection. This would probably be because it finds out the device name of +the job's controlling terminal and opens it, to prevent redirection. This +is hard to prevent in Solaris because there is no TIOCSTTY ioctl(). +Supposedly the same thing is accomplished by closing and reopening the slave +pty after doing setsid(). I added code to do this, but it made no +difference. (If I use lsz instead of sz, it is indeed redirected, but jams +up after about 15K.) ckupty.c, 27 Aug 2007. + +On Mac OS X with sz 3.73 1-30-03, however, the redirection works, so I +assume it would also work in Linux, FreeBSD, NetBSD, etc, too. Doing the +full test suite on Mac OS X: + + gkermit lrzsz rzsz + kst OK zst FAIL (1) OK + ksb OK zsb FAIL (2) OK + krt OK zrt OK (3) OK for 100K file, fails for longer. + krb OK zrb FAIL (4) OK (1MB all-bytes test pattern) + +(1) 64K file OK every time; 100K file fails every time. +(2) 10K file fails every time. +(3) Succeeds with 800K file but gets a few recoverable errors. +(4) Succeeds with 48K binary file with some errors, fails with longer ones. + +So actually it looks pretty good, it's just that lrzsz messes up. When +sending with lsz if I include -L 512 it sends the 100K test file with no +errors, but still chokes on longer ones. + +Testing on Mac OS X again, but this time over a clear-text Telnet connection: + + gkermit lrzsz rzsz + kst OK zst FAIL(1) OK + ksb OK zsb FAIL(2) OK + krt OK zrt OK(3) OK + krb OK zrb FAIL(4) OK + +(1) Almost worked, finished 777K out of 824K without errors. +(2) Got tons of errors, failed in first 30K out of 1000K. +(3) OK for 100K file but fails for larger. +(4) OK for 48K binary fail but fails for larger. + +Maybe see if we can do without the OPENPTY part. + +TOMORROW -- just clean up the code, add some SET / SHOW / HELP commands, +document it, and move on. + +Note: In K95, SET WINDOW sets the Zmodem packet length, 32 - 1024, multiple +of 64. + +SEE ~/80/external.txt + +Changed ftp port from int to unsigned int. ckcftp.c, 30 Aug 2007. + +Tried again to build KRB4/KRB5/SSL/TLS version for Solaris 9. Had to update +the build procedure again, of course, because of new file and directory +names, but ran into problems anyway because the +cu-solaris9g+krb5+krb4+openssl+shadow+pam+zlib target was calling another +target that did not know about the hardwired pathnames. Integrated the two +targets and tried building again. It actually compiled ok (but with lots of +warnings from the security modules), but failed at link time with +krb5_init_ets not found; fixed that with an #ifdef NO_KRB5_INIT_ETS, now it +builds OK but without the ftp client. Tried building it WITH the FTP and +that was OK too, no changes needed except to the build procedure. 12 Feb +2008, that is: C-Kermit 8.0.212 : 20080212. + +Tried to build with -DCK_SRP and -lsrp but: + + hash_supported ckcftp.o + hash_getdescbyname ckcftp.o + hash_getdescbyid ckcftp.o + cipher_getdescbyname ckcftp.o + krypto_delete ckcftp.o + krypto_new ckcftp.o + cipher_supported ckcftp.o + krypto_msg_priv ckcftp.o + krypto_msg_safe ckcftp.o + hash_getlist ckcftp.o + cipher_getlist ckcftp.o + cipher_getdescbyid ckcftp.o + +Sent mail to Tom Wu and backed off for now. makefile, 14 Feb 2008. +(Tom Wu never answered; seems like SRP is defunct.) + +The ".blah = xxx" form of variable assignment only worked for variables +names of length 22 or less, noticed and fixed by Wolfram Sang. ckucmd.c, +5 Mar 2008. + +In "set host /pty ssh ..." connections, the INPUT command suddenly stopped +working. This is in Solaris 9. It happens with all 8.0.* versions of +C-Kermit, so it's nothing to do with ttptycmd(). Added some debug() +statements but they don't show anything. Turns out there wasn't a problem +after all. Wed Mar 26 16:04:53 2008 + +Changed cmifi() to not print "?No files match" (or whatever) if SET QUIET ON. +ckucmd.c, 26 Mar 2008. + +Added \v(remoteip) for the IP address of the host we're connected to, +and \v(inmessage) for INPUT status messages corresponding to \v(instatus). +ckuusr.h, ckcmai.c, ckuus[24].c, 26 Mar 2008. + +Made \fkeywordval() strip braces/quotes from the right-hand side so we can +handle things like: + + password="stringwithspaceatend " + +ckuus4.c, 6 Aug 2008. + +Added invisible PUTENV command for UNIX only. Value should not be enclosed +in doublequotes. Requires lge \v(buildid) 20080826. ckuusr.[ch], 26 Aug 2008. + +Added SET VARIABLE-EVALUATION { RECURSIVE, SIMPLE }. This is highly +experimental, but also highly desirable if it works out. SIMPLE inhibits +the default recursive method of evaluating \%x and \&x[] variables, which +is, quite frankly, nuts and makes programming in Kermit at best +counterintuitive. I made an exception in the case of array subscripts, +because changing how they are evaluated could break a lot of scripts, and +anyway there should never be any harm in evaluating them recursively because +their final value is always (or should be) numeric, not some string that +might contain backslashes. The SET VAR setting is on the stack, just like +SET QUIET (it follows the quiet/xquiet code in ckuus[356].c), so macros or +command files that change it can't break the script that invokes them. +Added \frecurse() to force recursive evaluation of a \%x or \&x[] variable +regardless of the VARIABLE-EVALUATION setting. Added \v(vareval) to allow +programmatic setting to current setting. Tested on Solaris 9 but should be +totally portable. ckuusr.[ch], ckuus[356].c, 11 Sep 2008. + +From Günter Knauf: 64-bit builds were failing on SuSE Linux because +libresolv and libcrypt were in lib64 rather than lib; updated the tests in +the linux makefile target to find them. makefile, 12 Jan 2009. + +Tried building on Red Hat Enterprise Linux Server release 5.3 64-bit. +There is no curses or ncurses. "make linuxnc" compiled OK but collapsed at +link time looking for crypt(), res_search(), and dn_expand(). Turned out +the linuxnc (and linuxc) targets needed the same treatment as the Linux one +for 64-bit Linuxes. makefile, 3 Mar 2009. + +Consolidated the linux targets so we no longer need three separate ones for +curses, ncurses, and no curses. "make linux" works ok on computers with and +without (n)curses. "make linux+ssl", ditto. "linux+krb5+ssl builds OK but +needs -DNO_KRB5_INIT_ETS". Makefile, 3 Mar 2009. + +Fixed copyright date announced in herald, ckuus5.c, 3 Mar 2009. + +Patch from Seth Theriault to avoid deprecation warning for utmp references +in ckufio.c in Mac OS X 10.5 (later, this became a consolidated makefile +target that works automatically for at least Mac OS X 10.3.9 through +10.5.6). makefile, ckufio.c, 28 April 2009. + +zshcmd() (the function used by RUN and ! to run external commands) was not +falling back as expected in Linux RHEL4/5 if SHELL was not defined in the +environment. Also in all Unix versions, there was no indication if a RUN/! +command failed (other than the return code) because the specified shell +didn't exist or was not executable (e.g. the SHELL environment variable was +misdefined). Now it prints the name of the offending shell and the reason +it couldn't be executed (Not found, Permission denied, etc). ckufio.c, +28 April 2009. + +There is no easy way to get the last field of string; for example, the +extension from a filename, which might have any number of fields. In +general we want to be able to get "word number n" counting from the right; +\fword() lacks this ability. Now if you give it a negative word number, +that says to count from the right; for example \fword(one two three four +five, -2) returns "four". ckclib.c, ckuusr.c, 14 May 2009. + +Fixed a typo in the aix51+openssl (SSLLIBS should have been SSLLIB). +From Jason Lehr. makefile, 27 May 2009. + +Updated the linux+openssl+zlib+shadow+pam target to chain to the new main +Linux target. A bunch of other ones remain un-updated. makefile, 12 Jun 2009. + +Updates to the new Mac OS X 10.5 target from Seth Theriault (which is +supposed to work on all Mac OS 10-point-anything) to avoid warnings +that came up on on Mac OS 10.4.11/Intel. Once this one is proven we should +be able to remove/consolidate lots of other ones. makefile, 12 Jun 2009. + +C-Kermit disables SSL with the message "?OpenSSL libraries do not match +required version." if the version of OpenSSL that Kermit was built with is +not exactly the same as the version that is loaded dynamically at runtime. +This is actually the proper behavior, since APIs are not guaranteed not to +change between OpenSSL versions prior to 1.0.0. Made the error message more +informative. ck_ssl.c, 26 Aug 2009, and again 28 Aug 2009. + +AIX 6.1 is out, it is really just a new name for AIX 5.4. Added makefile +targets, plus for the first I made AIX 4.2 and later figure out its version +number in the makefile target so we don't have to keep adding new -DAIXnn +sections to the code, and also get its hardware name (e.g. "powerpc") from +uname at make time, rather than hardwiring "rs6000" as I did before. +Consolidated all AIX 4.2 and later targets so now just "make aix" or "make +aix+ssl" can be used. Except not the gcc ones as they have some quirks so +I'd rather not disturb them. Tested this on AIX 5.3. +makefile, 28 Aug 2009. + +From Kinjal Shah, a correction to the Linux makefile entry that allows it +find the 64-bit curses or ncurses library. makefile, 29 Aug 2009. + +Renamed aix4[23]: to oldaix4[23]: in makefile to fix the warning messages +I didn't notice before. I didn't want to remove them because they have +some special things that might still be needed, if anybody still has these +AIX versions. makefile, 29 Aug 2009. + +Built on RHEL 5.3 64-bit, regular and with OpenSSL 0.9.8e. 31 Aug 2009. + +Built on NetBSD 5.0.1/i386, regular and with OpenSSL 0.9.9-dev, 1 Sep 2009. + +Changed SSL message to mention LD_LIBRARY_PATH (Solaris), SHLIB_PATH (HP-UX), +LIBPATH (AIX), or LD_LIBRARY_PATH (Linux). ck_ssl.c, 3 Sep 2009 + +Noticed that "make linux+openssl" fails to include -lutil a link time, which +it needs for openpty(). That's because this target is obsolete. I renamed +it to be oldlinux+openssl and added linux+openssl as a synonym for +linux+ssl. makefile, 3 Sep 2009. + +Tested linux+openssl+zlib+shadow+pam, it's OK. Also linux+krb5. Also +linux+krb5+ssl. makefile, 3 Sep 2009. + +Tried building on Solaris 9 with OpenSSL 0.9.8k with +solaris9g+openssl+shadow+pam+zlib, it failed like so: + + ck_ssl.c:2875: error: conflicting types for 'inet_aton' + /usr/include/arpa/inet.h:52: previous declaration of 'inet_aton' was here + make[2]: [ck_ssl.o] Error 1 + make[2]: Leaving directory hmt/sirius1/prv0/kd/fdc/solaris9ssl' + make[1]: [solaris2xg+openssl+zlib+pam+shadow] Error 2 + make[1]: Leaving directory hmt/sirius1/prv0/kd/fdc/solaris9ssl' + make: [solaris9g+openssl+shadow+pam+zlib] Error 2 + +The problem was caused by including an inet_aton() function ck_ssl.c for +the benefit of platforms that don't have one in their libraries. This is +defeated by including NO_DCL_INET_ATON in KFLAGS. I added this, but then +I thought it would be a good idea to automatically sense the OpenSSL +version so we can automatically set OPENSSL_097 or OPENSSL_098 rather than +bombing out, so I added code to do that too, and also to set the Solaris +version number: 9, 10, or 11. The new entry is solaris9g+openssl. +ckcdeb.h, makefile, 3 Sep 2009. + +Fixed a complaint in ckufio.c about implicit declaration of initgroups. +ckufio.c, 4 Sep 2009. + +Built on Solaris 10 with gcc and Sun CC using new solaris{9,10,11} target +that is like the new solaris{9,10,11}g one but without the gccisms. +makefile, 4 Sep 2009. + +Changed solaris{9,10,11}g+ssl target to set only the SSL-specific things and +then chain to the main solaris{9,10,11}g target. Tested OK on Solaris 9 and +10. makefile, 4 Sep 2009. + +Created solaris{9,10,11}+ssl target that is exactly like the +solaris{9,10,11}g+ssl except it chains to the solaris{9,10,11} target +instead of the solaris{9,10,11}g one. That is, it builds an SSL version of +C-Kermit using Sun CC rather than gcc. makefile, 4 Sep 2009. + +Tried building on HP-UX 10.20, bundled (non-ANSI) compiler ("make +hpux1000"). This failed until I: + + . Moved a struct initialization out of setextern(), ckuus3.c. + . Removed an ANSIism from the declaration of sigchld_handler() in ckutio.c + . Added a cast to strcmp() in zvuser(), ckufio.c. + +Builds OK now. Built OK with "hpux1000o" (the ANSI compiler) too. +And with "hpux1000gcc". Couldn't test "hpux1000o+openssl". 21 Sep 2009. + +The Sony Playstation 2 and 3 are 64-bit PowerPC platforms that can run Linux +if it is installed as an "other OS" on its hard disk; and the Linux kernel +since 2.6.21 supports the PS3 without any patching required. Pawel Rogocz +reported that "make linuxppc" (one of the old targets that has not yet been +integrated into the main "linux" target) compiles OK on 2.6.29-ydl61.3 +(Yellow Dog Linux release 6.2 'Pyxis'), but fails at link time because +'openpty' isn't found, because -lutil was not included, because that part +was added only to the main linux target. I asked him to try "make linux" +and he sent back a transcript in which there were thousands of errors from +the curses code ckuusx.c. Later I tried it myself and it built without a +hitch. My theory is that between then and now, a missing piece of the +ncurses library (/usr/include/ncursesw) was installed. 21 Sep 2009. + +HP-UX 9.05 on PA-RISC 9000/712 building with hpux0900 (bundled compiler): + . ckutio.c compilation failed with PENDIN and FLUSHO not defined in + pty_make_raw(). I dummied definitions for them to handle this situation + on this or any other platform where it might crop up. + ckutio.c, 24 Sep 2009. + . Ditto for the PTY module, + IMAXBEL. ckupty.c, 24 Sep 2009. + . References to endusershell() were fatal in the bundled compiler. Changed + the hpux0900 target to define NODCLENDUSERSHELL, and put a special case + in ckufio.c to not put a cast in front of the call if NODCLENDUSERSHELL + is defined. Now it builds and links OK. makefile, ckufio.c, 24 Sep 2009. + +HP-UX 9.05 on PA-RISC 9000/712 building with hpux0900o (optimizing compiler): + . Warnings in ckutio.c at line 14860 about arguments to select (pointers + are not assignment-compatible). "man select" says arguments are ints. + Defining INTSELECT fixes these warnings but results in fatal errors later + around line 14881 and others in the area involving FD_SET. This was too + involved so I put it back as it was. 24 Sep 2009. + +Built OK on Solaris 10 with Sun CC. A couple warnings about implicit +function declarations for curses routines because apparently they aren't +declared in curses.h. Tuff. 25 Sep 2009. + +Tried building on Solaris 10 with Sun CC and OpenSSL 0.9.8k, and this +uncovered various loose ends in the solaris9+openssl target, which I fixed. +makefile, 25 Sep 2005. + +Fixed four typos in printfs in ck_ssl.c, \% instead of just %. 25 Sep 2009. + +Squelched 20-some complaints about a character array being referred to +directly instead of by a pointer, plus several other similar nits to get rid +of all the compilation warnings on Solaris 10 with Sun C 5.8 Patch 121015-06 +2007/10/03. ckctel.c, ckctel.h, 25 Sep 2009. + +Built the result on the same Solaris 10 system with gcc 4.2.4 using the +new solari10g+openssl target, working out a few kinks here too. +makefile, 25 Sep 2009. + +Made consolidated Solaris 9/10/11 64-bit targets for gcc, solaris9g64, +solaris10g64, solaris11g64, tested on Solaris 10 Sparc. makefile, 25 Sep 2009. + +Made consolidated Solaris 9/10/11 64-bit targets for Sun cc: solaris9_64, +solaris10_64, solaris11_64. These simply set a couple flags and chain to +the main solaris9 target. makefile, 25 Sep 2009. + +Removed a bunch of old superfluous Solaris 9 and 10 targets: oldsolaris9, +oldsolaris9lfs, solaris9g64 solaris9g_64, oldsolaris10 old solaris10lfs, +oldsolaris10+openssl, oldsolaris10g+openssl, solaris10_64, oldsolaris10g, +solaris10g_64, solaris10g64. There are still plenty more to prune but it's +a start. makefile, 25 Sep 2009. + +Added or fixed some missing prototypes in ckctel.h: +fwdx_send_xauth_to_xserver(), fwdx_parse_displayname. 25 Sep 2009. + +Improved the instructions for building secure versions in the makefile, +using this example: + + make solaris9+openssl "SSLINC=-I/opt/openssl-0.9.8k/include" \ + "SSLLIB=-L/opt/openssl-0.9.8k/lib" + +makefile, http://kermit.columbia.edu/security.html, 25 Sep 2009. + +Built on HP-UX 11.11, 26 Sep 2009: + . make hpux1100 (ok) + . make hpux1100gcc (ok) + . make hpux1100o (gets a lot of warnings about sendpath and sendfile, + because they are also declared in , but builds OK) + . make hpux1000gcc+openssl \ + SSLINC=-I/opt/openssl/include SSLLIB=-L/opt/openssl/lib + +Note: sendpath and sendfile are not Kermit symbols. The warnings are coming +from socket.h: 'Redeclaration of "sendfile" with a different storage class +specifier'. This is nothing new; see notes of 2-4 Jan 2005. + +From Peter Eichhorn: + . Update to makefile to make current code build OK on HP-UX 8.00. + . Changes to format of some hints to make them more copy-and-pastable. +makefile, ckuu5.c, 28 Sep 2009. + +From Peter Eichhorn: Changes to HP-UX 7.0 target to increase the switch table +stack size, which was overflowing. makefile, 30 Sep 2009 + +HP-UX 6.5 (1989), "make hpux0650tcpc"... (8:19...) Needed to not include +arpa/inet.h (which doesn't exist) and not use host address lists (add +-DNOHADDRLIST), which gets us past ckcnet.c, but in ckcftp.c we bomb out on +FD_SETSIZE undefined. Somehow we worked around this in ckcnet.c. Patched +in a definition in ckcftp.c, and also added -DINTSELECT to compiler flags. +Compiles ok, bombs at link time on bcopy, bzero, FD_ZERO, FD_SET, FD_ISSET. +Now it compiles and links OK but dumps core when started. Added +-DNOCKGETFQHOST, rebuilt from scratch (takes 35 minutes). It starts OK, but +it dumps core when given a "telnet xxx" command, where xxx is a hostname. +However, it works OK if an IP address is used: "telnet 123.45.6.78". It +took all day to track this down, but now it's fixed (see the #ifdef HPUX6 +sections of ckcnet.c). So now (for the first time, I think) we have both +telnet and ftp in HP-UX 6.x, if anyone cares. ckcnet.[ch], ckcftp.c, +makefile, 2 Oct 2009. + +Changed default SET TERMINAL TYPE type for K95 from vt320 to vt220. This is +because Unix OS's such as Solaris have dropped vt320 as a terminal type. +settrmtyp(), ckuus7.c, 5 Oct 2009. + +I moved the PUTENV command code, which was inline, to a function, doputenv(). +ckuus[r7].c, ckuusr.h, 5 Oct 2009. + +Changed the UNIX version of SET TERMINAL TYPE to take a value and then do +the equivalent of "export TERM=value" by calling doputenv(). This sets +\$(TERM) correctly and passes its value along to inferior processes. +However, to make this take effect within Kermit itself (for the fullscreen +file transfer display and for the SCREEN command, Ctrl-L, etc) I also had to +reinitialize the curses database, which is tricky because normally if you +feed it an unknown terminal name, it just exits. ckuus7.c, 5 Oct 2009. + +Changed the little-known and little-used RESET command (which closes all +open files) to also put command echoing back to normal in case it got +messed up somehow (as in HP-UX 6.5, upon returning from PUSH). +ckuusx.c, 5 Oct 2009. + +For Unix, increased string buffer sizes for wildcard expansion for all +platforms that have BIGBUFOK defined from 500000 (0.5M) to 10000000 (10M) +bytes, and for 64-bit builds to 2000000000 (2G) bytes. No point making +it bigger than that because malloc's argument is a size_t, which is an int. +ckufio.c, 5 Oct 2009. + +Built on Mac OS X 10.4.11, required one minor adjustment to the makefile +(-DNODCLINITGROUPS). This was using the macosx10.5 target, which is +supposed to be universal like the linux and netbsd targets, but not yet +proven. Also built a 64-bit version (-mpowerpc64 -mcpu=G5 -mtune=G5 +-arch ppc64); it compiles and links OK but won't start: "Bad CPU Type +in executable". Fix later... makefile, 5 Oct 2009. + +Changes from Seth Theriault to suppress signed vs unsigned char warnings in +Mac OS 10.5.8 from gcc4, and a new makefile target for Mac OS X (presumably +10.3.9 or later) + Kerberos 5 and OpenSSL. ckutio.c, ckuath.c, ckctel.c, +ckcnet.c, ckcftp.c, ck_crp.c, makefile, 6 Oct 2009. + + Later I had to back out of these, because although it made for a + clean build, in the resulting executable SSL connections didn't work. + +Tue Oct 6 17:23:27 2009 +FTP address resolution is broken, but ftp_hookup() hasn't changed. +So... (see the #ifdef HPUX6 sections of ckcnet.c) (I did, and I rolled +back some of the changes from the other day, but it made no difference.) +Putting back the ckcftp.c from a few weeks ago makes no difference. +Putting back the ckcnet.c from a few weeks ago makes no difference. + +Added patches from Seth Theriault so macosx10.5+krb5+openssl would build +on Mac OS X 10.3.9. makefile, ckcftp.c, 7 Oct 2009. + +Built today's code on Linux RHEL4, NetBSD 5.0.1, Solaris 9, and Mac OS X +10.4.11, both with and without SSL. The NetBSD system has OpenSSL 0.9.9-dev. +7 Oct 2009. + +In Mac OS X 10.6, the following symbols are unresolved at link time: +_des_key_sched, _des_new_random_key, _des_ecb_encrypt, +_des_init_random_number_generator, _des_fixup_key_parity. This is +with OpenSSL 0.9.8k. But it doesn't happen on other platforms that +have 0.9.8k. + +Added SET SESSION-LOG NULL-TERMINATED-TEXT. This is for the benefit of a +speech synthesizer that will speak a line of text only after receiving a +NUL character. A more general solution would be to define a filter or +whatever, but who has time. ckuus[23x].c, 7 Oct 2009. + +Consolidated Mac OS X targets, and removed experimental 64-bit ones, because +they never could work in 10.5 and earlier because 64-bit libs are missing, +and 10.6 and later are 64-bit automatically. makefile, 8 Oct 2009. + +Built on Mac OS X 10.6.1. It came out automatically as a 64-bit build +because __LP64__ is defined somewhere that I can't find. But this explains +why the 0.9.8k on 10.6 comes up with missing symbols when the 0.9.8k lib +10.5 (or on Solaris or on Linux) does not: it's a different library: "Mach-O +64-bit dynamically linked shared library x86_64", rather than "Mach-O +dynamically linked shared library ppc". Probably the 64-bit version has +some things #ifdef'd out. Added -m32 to the CFLAGS and LNKFLAGS for the +macosx+krb5+openssl targets, and it built OK one time. But then the errors +came back. makefile, 8 Oct 2009. + +Updated C-Kermit installation for Mac OS X in ckuwr.html on the website. +8 Oct 2009. + +Tried some things to get around the problem with OpenSSL in Mac OS X 10.6, +to no avail. Asked Jeff. He said, "MacOS X no longer includes DES anywhere +on the system. Not for SSL, not for Kerberos, not for anything. This will +increasingly become the situation on new operating systems. Windows 7 and +2008 R2 will also ship with no DES." Sure enough, the Mac OS X Server +Upgrading and Migrating document for 10.6 says, "Mac OS X Server v10.6 does +not support single DES encryption. It supports AES 128 and 256 encryption +types. However, during a migration or upgrade from v10.4 to v10.6, servers +that were Kerberized by the v10.5 Open Directory server will not use the AES +128 or 256 encryption types. To use the AES 128 or 256 encryption types you +must re-Kerberize all servers." 12 Oct 2009. + +DES and 3DES encryption can be excluding removing the -DCK_DES flag. I +removed this one and -DLIBDES (and -m32) and this makes a working 64-bit +version. Then I added code to the macosx+krb5+openssl target to use these +flags if the Mac OS X version was 10.5 or less and leave them out for 10.6 +or later. Tested on 10.4.11 and 10.6.1. A better way to do it might have +been "nm -gj libssl.dylib | grep des_", but that gives the same results on +10.4 and 10.6. Also, 10.6 still has /usr/include/ssl/des.h. +makefile, 13 Oct 2009. + +Next issue: + In file included from ckutio.c:15674: + /usr/lib/gcc/i386-redhat-linux/3.4.6/include/varargs.h:4:2: #error "GCC no + longer implements ." + /usr/lib/gcc/i386-redhat-linux/3.4.6/include/varargs.h:5:2: #error "Revise + your code to use ." + +The problem occurs when trying to force a non-ANSIC build with GCC. +Changing the source file to include instead of +doesn't help because evidently requires an ANSI C compiler. +Nothing can be done about this. 13 Oct 2009. + +Next issue: Can't compile ckcftp.c with -DNOCSETS or -DNOSPL; some +#ifdef/#endif doesn't match up. Sigh, this is the hardest kind of thing to +debug. There's 17,622 lines of code in this module and no tool that I know +of.... Wait, I wrote one. But it shows all the #if/#ifdef/#ifndef's and +#endifs matching up just fine. Backing off to ckcftp.c of a few days ago +(before char / unsigned char casts were added), I see that it builds OK, so +I backed off to that one, but put back the special case #ifdef for MACOSX103 +declaring CONST gss_OID_desc, and it builds OK (the other stuff was purely +cosmetic, when will I learn?). ckcftp.c, 13 Oct 2009. + +Protected cvtstring() and related functions with #ifdef NOCSETS..#endif, +and ditto for the character-set conversion code in dorename(). +ckuus6.c, 13 Oct 2009. + +Fixed an #endif /* TNCODE */ that was a line too low in ttptycmd(), +causing -DNONET builds to fail. ckutio.c, 13 Oct 2009. + +There was a reference to doputenv() that wasn't guarded by #ifndef NOPUTENV, +fixed in ckuus7.c, 13 Oct 2009. + +Moved doputenv() and settermtyp() out of an #ifdef NOLOCAL section because +these are useful even when not making connections. ckuus7.c, 13 Oct 2009. + +Moved havelfs declaration outside of #ifdef NOXFER because it was also used +for other things. ckcmai.c, 13 Oct 2009. + +COPY /PRESERVE depended on code from the Kermit protocol module, which +is omitted in -DNOXFER builds. Disabled COPY /PRESERVE in -DNOXFER +builds. ckuus6.c, 14 Oct 2009. + +SHOW PROTOCOL code for external protocols had to be #ifdef'd out for +-DNOPUSH builds. ckuus4.c, 14 Oct 2009. + +There was some confusion between "No XYZMODEM" and "No external protocols"; +cleared up in ckuus3.c, 14 Oct 2009. + +After all that, 86 different combinations of feature selections built OK on +Linux. And the Kerberized version (K5) works OK on Linux for Telnet and FTP. +14 Oct 2009. + +Changed version number to 9.0. All modules, 16 Oct 2009. + +Need to make LOG SESSION log to a tty. Right now "log session +/dev/ttyKeySerial1" says "Write permission denied" even though the device is +crw-rw-rw-. This happens in zchko(), which is called by cmofi(). The +problem is that /dev/ is not writeable. I added a Unix-only clause that +attempts to open the file for write access using open(), in order to get a +file descriptor, which then can be passed to isatty() to check if it's a +tty, and if so, to allow access. And then close it. I tested this on Mac +OS X as follows: + + log session /dev/ttyKeySerial1 + telnet somehost + +The Mac's serial port was connected to the serial port of another computer +where Kermit displayed the incoming characters in CONNECT mode. Glitches: + + 1. The port has to be set up as desired in advance, outside of Kermit. + 2. log session /dev/ttyKeySerial1 will hang if any required modem signals + are not present when the port is opened. + 3. Bypasses lockfile mechanism - so we do this only if -DNOUUCP. + +For (2), I tried setting O_NDELAY / O_NONBLOCK, and this allowed zchko() to +continue, but then it freezes in the subsequent fopen(). So I changed +zopeno() to also check if the device is a serial port, and if so, to open() +it with O_NDELAY / O_NONBLOCK, and then convert the file descriptor into a +file pointer with fdopen(). + +Now for the speaking device that needs lines to be terminated by NUL... + + set session-log binary <-- need to put these in SHOW LOG + set session-log null-padded (and in HELP SET LOG) + set line /dev/ttyKeySerial1 + +This part works. + +This feature is enabled only for -DNOUUCP builds because serial ports aren't +like other Unix files; we would have to create a lockfile, but we can't do +that... actually, ttlock() takes a name as an argument, but ttunlck() does +not, so there would be no way to remove the lock. Anyway, there is only one +API for configuring the port (speed, flow control, etc) and it only works +with the SET LINE device, not any random file. To fix this would require +massive redesign and changes. ckuus[23].c, ckufio.c, 19-20 Oct 2009. + +I made -DNOUUCP the default for Mac OS X, since everybody winds up building +it that way anyhow. To undo this, do "make macosx KFLAGS=-UNOUUCP". +makefile, 21 Oct 2009. + +Changed SET SESSION-LOG TEXT to strip out ANSI escape sequences; +previously there wasn't that much difference between TEXT and BINARY logs. +It's still not perfect; for example it doesn't delete characters that the +user erased. (Made sure this still builds with -DNOESCSEQ.) +ckucns.c, 22 Oct 2009. + +Changed SHOW LOG to show the SET SESSION-LOG settings, as well as +SET DEBUG, which was not shown before. ckuus5.c, 22 Oct 2009. + +If a series of PUTENV commands is given, each new one undoes the previous +one, so only the last definition is seen by the new fork (or by Kermit +itself). Turns out you can't feed automatic variables to putenv(); they +have to be static, so to allow for multiple PUTENV commands Kermit has to +maintain an array of static strings. ckuus7.c, 6 Nov 2009. + +From Seth Theriault, a better way for the makefile to determine the +Mac OS X version number; there's a program for this, sw_ver. makefile, +6 Nov 2009. + +Peter Eichhorn reported that file-transfer failure hints were not coming +out since Dev.27. The only change I made since then was to skip them if +the file-transfer protocol was not Kermit. I was using the wrong variable +in the tests, 'proto' instead of 'protocol'. ckuus5.c, 6 Nov 2009. + +Changed Mac OS X targets to correctly extract the Mac OS major version +from uname -r in order to choose correctly between utmp and utmpx; this +wasn't working in 10.6.1. makefile, 6 Nov 2009. + +Fix from Seth T. for an oversight in the previous edit. Also add +MACOSX103 to "show features" display. makefile, ckuus5.c, 10 Nov 2009. + +Added REJECT as a synonym for DISCARD in SET FILE COLLISION; it's more +intuitive and more accurate. ckuus[27].c, 15 Nov 2009. + +\fsplit() and \fword() always break on 8-bit characters unless you explicitly +put every single 8-bit value into the include set, e.g. (for a TSV file): + + undef include + for \%i 128 255 1 { + if == \%i 9 continue + .include := \m(include)\fchar(\%i) + } + .\%n := \fsplit(\m(line),&a,\9,\m(include)) + +I changed cksplit() to treat all 8-bit bytes 128-255 as non-break characters +by default. It might have made more sense to do this for 160-255 (since +128-159 are traditionally C1 control characters) but thanks to Microsoft +tradition is out the window. To treat one or more 8-bit characters as break +characters, put them in the break set. This might break some scripts, but I +doubt it because this flaw was so awful that if anyone had come up against +they would have let me know. ckclib.c, 16 Nov 2009. + +Changed the netbsd target to set -funsigned-char, since cc on NetBSD is +actually gcc. makefile, 16 Nov 2009. + +Changed macosx targets to get the CPU type from the HOSTTYPE environment +variable. Also added getenv("HOSTTYPE") as a last-resort method to set the +\v(cpu) variable at runtime (maybe it should be the first resort?)... +ckuus4.c, makefile, 16 Nov 2009. + +Made sure the solaris9_64 and solaris10 targets still work. 16 Nov 2009. + +Made sure the current source package builds OK on HP-UX 10.20... Got a lot +of "warning 6062: Optdriver: Exceeding compiler resource limits in xxx; some +optimizations skipped. Use +Onolimit if override desired" but it builds OK. +Tested long file transfer; works OK. 17 Nov 2009. + +Built on FreeBSD 7.2 with and without OpenSSL, all OK. 17 Nov 2009. + +Built on NetBSD 5.0.1 with and without OpenSSL, all OK, but netbsd+krb5 +fails with "can't find -lgssapi_krb5"; worked around this with +"K5LIB=-L/usr/local/kerblib" (where the lib actually is on this host) but +then it failed with "ckcftp.c:13868: error: 'gss_nt_service_name' undeclared". +17 Nov 2009. + +I found a VMS 6.2 system... Takes a loooong time to build there. In +ckuusy.c, DEC C didn't like the prototypes and declarations of dorlgarg() +and dotnarg() as static so I made them not static. But that didn't help, +now it fails at the very end, saying the final #ifdef is an invalid +statement. It looks like an #ifdef mismatch that affects only VMS. I ran +my #ifdef matcher, it turned up nothing. I substituted a copy of ckuusy.c +from 2007, it comes up with the same errors. Then I substituted the copy +from 8.0.211 from 2004, and this one compiled OK and, miraculously, the +whole mess even linked OK and runs OK. The Alpha binary is 2.84MB. Now I +have 4500 lines of code to compare.... I went through the two files line by +line and I can't see a single thing wrong. I gave up and tried building the +TCP/IP version. It builds fine except for ckuusy.c, with the utterly +useless error message: + + #endif /* NOCMDL */ + ...................^ + %CC-E-BADSTMT, Invalid statement. + +Indicating the last line in the file. Just for the heck of it, I put +another line after that one: + + /* This is a test */ + +and got: + + /* This is a test */ + ....................^ + %CC-E-BADSTMT, Invalid statement. + +So it is not objecting to anything in the file. Trying the old LISP trick, +I put an extraneous closing bracket after that. Success! Honestly, I don't +see anything wrong with file. It's DEC C V5.3-006. I suspect a C bug. +I'll leave it like this for now until I get access to some other VMS +versions. Another clue is that when building the network version I get a +horrible warning I never saw before from a module that hasn't been touched +in a very long time (ckvrtl.c). Also, in the network version, I note that +the FTP code is not compiled in. We have to try this again with some +command-line switches, but it'll do for now. ckuusy.c, 18 Nov 2009. + +---C-Kermit 9.0 Alpha.01--- + +From Steven Schweda (SMS), the real solution for the VMS closing brace +problem, it wasn't a DECC bug, it was a me bug. ckuusy.c, 20 Nov 2009. + +Rediscovered the new VMS build options: f for Long Files, i for Internal +FTP. "make mnf" doesn't work on VMS 6.2, it looks like the VMS definition +for CK_OFF_T got lost. Same thing with "make mfi". Come back to this later. + +From Gerry Belanger, a fix to INPUT /COUNT:n. ckuus4.c, 26 Nov 2009. + +Added \fsqueeze(s), returns string s with leading and trailing whitespace +removed, Tabs converted to Spaces, and multiple spaces converted to single +spaces. For now, ASCII only, no options. ckuusr.h, ckuus[24].c, 27 Nov 2009. + +I wrote a Kermit script to read a big file of addresses on Solaris 9, +\fsqueeze()ing each line. After about 14000 lines, there was a malloc +failure in getnct() (the command-file reader). There's nothing wrong with +\fsqueeze(), the failure is on a deeper level, because the same thing +happens if I use \fupper() (which is structurally identical to \fsqueeze()) +in the same script. The problem is not in getnct() either, because every +malloc() is freed (I checked). On the other hand, the same script (with +\fupper() instead of \fsqueeze() completes OK in C-Kermit 8.0.201. If I +remove the function call (\fsqueeze() or \fupper()) from the script, it also +runs OK in 9.0. This seems to point the finger at fnevel(), which contains +countless malloc's and free's. But comparing fneval() between 8.0.211 and +9.0, I don't see any difference that would explain this behavior -- nothing +at all that involves malloc(), makstr(), or free(). Nor any pertinent +change in the caller (zzstring) of fneval(). 27 Nov 3009. + +Another problem is that when this happens, the error is not caught (e.g. by +the IF FAIL statement after the command that contains the function call); +instead, C-Kermit returns immediately to its prompt. 27 Nov 2009. + +It could simply be that some of the buffers we allocate are much bigger now. +But again, I don't see much difference between 8.0.211 and 9.0; we were +already allocating 32K command-related buffers (malloc() takes a size_t, and +size_t is an int almost everywhere). I built the same source on NetBSD and +ran the same script (with \fqueeze()), and it worked fine. Let's worry +about this later, if it comes up. 27 Nov 2009. + +Built OK on Silicon Graphics IRIX 6.5 R10000; regular build OK, SSL and +Kerberos builds failed. 30 Nov 3009. + +Tried to build on Digital Unix 4.0F but it blew up in ckutio.c, apparently +not recognizing any of the terminal struct symbols from termios.h. Tried +again with gcc, same thing. Tried explicitly #including +within #ifdef TRU64, same thing. What could have changed? 30 Nov 2009. + +Built OK on Linux RHEL5.4/Itanium-2, make linux. The secure build +required "FLAGS=-DNO_KRB5_INIT_ETS" and built OK. 30 Nov 2009. + +Built OK on Digital Unix 4.0F using "make osf" instead of "make tru64-40f". +I don't know why the specific target doesn't work, but it's not worth +chasing down. 2 Dec 2009. + +Built OK on MirBSD 10, despite a lot of gratuitous compiler warnings. Built +OK on MirBSD 10, OpenBSD 4.5, and Fedora 10. 3 Dec 2009. + +(Various other successful Unix builds in these weeks...) + +Built on VMS 7.2 and 8.3 with and without TCP/IP, no problems. 11 Jan 2010. + +Built on VMS 8.3 with "make fi" to include the FTP client and long-file +support (mid Jan 2010). + +Built on VMS 8.3 with UXC 5.6 and HP SSL 1.3, which is OpenSSL 0.9.7e. +It compiled and linked OK but when I tried to make an FTP SSL connection +it crashed in SSL$LIBSSL_SHR, which is called from ssl_auth(), after having +had TLS accepted as an authentication type, but before actually +authenticating. In Unix: + + 19. ftp open ftp.somecompany.com /user:pge.com/test_quota /password:xxxxxx +Connected to ftp.somecompany.com. +220-Somecompany FTP v6.0 for WinSock ready... +220 Welcome to the online storage FTP server. Please check the main web +site for system announcements and AUP. (O) +---> AUTH TLS +234 AUTH command OK. Initializing SSL connection. +TLS accepted as authentication type +SSL DEBUG ACTIVE +=>START SSL/TLS connect on COMMAND + +In VMS: + + 19. ftp open ftp.somecompany.com /user:pge.com/test_quota /password:xxxxxx +Connected to ftp.somecompany.com. +220 Somecompany FTP v6.0 for WinSock ready... +---> AUTH TLS +234 AUTH command OK. Initializing SSL connection. +TLS accepted as authentication type +SSL DEBUG ACTIVE +%SYSTEM-F-ACCVIO, access violation, reason mask=04, virtual +address=FFFFFFFF8001A120, PC=000000000068B118, PS=0000001B + +Note: The Unix version received the second 220 response, the VMS version did +not. That's odd, it's the same code... 25 Jan 2010. + +Added some essential details to the HELP FSEEK text. ckuus2.c, 25 Jan 2010. + +Discovered that the result returned by \fsearch() is totally unreliable. +This is probably too hard to fix. + +FSEEK did not pay attention to SET CASE, searches were always case sensitive. +Fixed in ckuus7.c, 26 Jan 2010. + +FSEEK failed to find anything if the search pattern was matched in the first +line of the file. Fixed in ckuus7.c, 26 Jan 2010. + +\fword() and \fsplit().... Another change, but not backwards-incompatible. +One may now put the word ALL (just like that, all uppercase) as the include +set (4th argument) to indicate that there will be no break characters other +than those explicitly given in the break set, e.g. \fsplit(\m(xx),&a,:,ALL) +breaks a line only on a colon (:), nothing else. The original rules for +cksplit() were more than a little counterintuitive: the default break set is +all non alphanums, and the default include set is all alphanums, so if you +wanted to parse (say) a CSV file, breaking only on comma, you had to think +of all the characters you wanted to keep. This way you just say ALL. +ckclib.c, 26 Jan 2010. + +Speaking of CSV files... How can you put comma as a function argument when +comma is the function-argument separator? Use one of these forms: + + \fsplit(\m(xx),&a,",",ALL) + \fsplit(\m(xx),&a,{,},ALL) + \fsplit(\m(xx),&a,\44,ALL) + \fsplit(\m(xx),&a,\fchar(44),ALL) + +From John Dunlap, U. of Washington Applied Physics Lab: 'When "stty -a < +/dev/ttyS0 | grep crtscts" shows "crtscts" (not "-crtscts") and when using a +three wire serial interface and when asking kermit to not use flow control +(set flow none) then "ckutio.c1" (see attachments) fails while "ckutio.c" +works. The result of "diff -u ckutio.c1 ckutio.c" is attached as "diffs"'. +ckutio.c, 26 Jan 2010. + +Changed the year from 2009 to 2010 in the modules I worked on today and in +the heralds, etc. ckckmai.c, ckuus5.c, ckutio.c, ckclib.c, ckuus7.c, +26 Jan 2010. + +Built on Linux Fedora Core 3, regular and with OpenSSL 0.9.7a. Built on +Ubuntu 9.4 OK, but SSL and Kerberos builds failed due to not finding libs +and/or header files. I'm sure this could be fixed... 27 Jan 2010. + +Added SSL, KRB4, and KRB5 to the startup herald for versions that were +built with SSL, Kerberos 4, or Kerberos 5. Built OK on Fedora 3 with +linux+krb5+ssl and new banner shows correctly. ckuus5.c, 27 Jan 2010. + +Set NO_KRB5_INIT_ETS by default in ckuath.h since krb5_init_ets() is a no-op +in Kerberos 1.4.x and later and in some installations it can't be found, +which clobbers the build. ckuath.h, 27 Jan 2010. + +Adapted to MINIX 3 1.5, the first version that has virtual memory according +to Andy T, who should know. On earlier versions (e.g. MINIX 3 1.2) any +attempt to build C-Kermit causes the compiler to crash. Now the compiler +doesn't crash but it spews out countless warnings about old-fashioned +function declarations that I don't get anywhere else. The real problems +came in ckutio.c where numerous symbols were undefined at compile time and +the POSIX function tcgetpgrp() was not found at link time, even though there +is a prototype for it in the MINIX header files, and there is no alternative +(since POSIX doesn't let us use ioctl()). Also note that there is some +confusion over the compile-time symbols MINIX, MINIX2, MINIX3, and MINIX315. +You would expect MINIX to mean "any version of MINIX" but in some parts of +ckutio.c it means MINIX 1.0. I sincerely doubt that C-Kermit 9.0 can be +built on any version of Minix before 3.1.5 so I removed the confusion and +made MINIX mean "any Minix". It builds on 3.1.5 OK now, except for the FTP +client. This can probably be fixed but... Modules changed: ckcdeb.h, +ckuver.h, ckcmai.c, ckuus5.c, ckutio.c, 1 Feb 2010. + +Later.. Andy says MINIX does not support job control, so no program is ever +in the background. That settles that! 1 Feb 2010. + +Built OK on Minix, Linux, Mac OS X, Solaris 9, NetBSD 5.0.1... 1 Feb 2010. + +---C-Kermit 9.0 Alpha.02--- + +From Christian Corti at Uni-Stuttgart.de: fixes to allow building on SunOS +4.1, which once was my main development platform but which is long-gone from +here. ckupty.c, ckutio.c, 9 Feb 2010. (He says it is also necessary to +comment out the "struct winsize" and "struct ttysize" in sys/ioctl.h; +otherwise there will be a conflict with sys/ttycom.h (included by termios.h) +which also declares these structs. But you need both includes.') + +From John Dunlap, a fix for Kermit protocol fixed packet-timeout interval +going to a unexpected value (missing else clause in two places). +ckcfn2.c, 9 Feb 2010. + +Added an aixg target to build on AIX with gcc when gcc is not installed as +cc, and also added CC=$(CC) CC2=$(CC) clauses to the aix and aix+ssl +targets. Wow, AIX really loses bigtime when receiving files through its ssh +server. Streaming can't be used, sliding windows recover from errors but +there are tons of them using the default 4K packets; 500 works much better. +Built with IBM cc and gcc, and also tested (successfully) the new aix+ibmssl +target, in which the OpenSSL headers and libs are in a standard place. +makefile, 9 Feb 2010. + +In ckupty.h, make the #include be #ifndef SUNOS41. +From Christian Corti. 10 Feb 2010. + +Built on VMS E8.4. 12 Feb 2010. + +Tried to build on a real VAX-11/785 but the machine seems to be seriously +wedged. 12-15 Feb 2010. + +Added note to CKVKER.COM to the effect the the 'f' option has no effect +on VAX architecture. 15 Feb 2010. + +Moved the #include "ckvrtl.h" in the FTP module to below the include for +utime.h, because building the VMS version with the 'i' option (meaning +"include internal ftp client") results in "struct utimbuf tp" erroring out +because struct utimbuf is not defined yet (at least in some version of VMS +with some version of C). From Rob Brown, ckcftp.c, 20 Feb 2010. + +From Martin Vorlaender: new code in VMS C-Kermit build procedure to detect +OpenSSL version automatically. ckvker.com, 22 Feb 2010. + +Added code to INPUT command to strip ANSI escape sequences. It's activated +by SET SESSION-LOG TEXT. ckuusr.h: added prototype for chkaes(); +ckucon.c, ckucns.c: made inesc[] and oldesc[] global instead of static; +ckuus4.c: doinput() code for skipping escape sequences. 1 Mar 2010. + +Peter Eichhorn complained that if you make an ssh connection with Kermit, +then log out from the ssh host, and then use a "connect" command to +make a new connection to the same host (which you can do with Telnet), +Kermit says (e.g.): + + DNS Lookup... Can't get address for ssh -e none somehostname + Sorry, can't open ssh -e none somehostname: Error 0 + +I added code to detect and handle this case and it seems to work OK, even +though it's kind of a hack. ckuusr.[ch], ckuus7.c, 1 Mar 2010. + +There has never been a clean way to put debugging messages (ECHO commands) +in a script which are executed only if debugging is desired and ignored +otherwise. You'd have to set a random variable and test it, or define a +macro or whatever. To make this more straightforward, I added SET DEBUG +MESSAGE ON/OFF/STDERR, and added a new MESSAGE (syn: MSG) command for printing +debugging messages to stdout if SET DEBUG MESSAGE is ON or to stderr if SET +DEBUG MESSAGE is STDERR. ckcmai.c, ckuus[r23].c, 12 Mar 2010. + +Also for debugging and error messages, I added \v(lastcommmand) so that +the command that failed can be included in an IF FAIL or DEBUG error message. +This works even for commands that have syntax errors. +ckuusr.h, ckuus5.c, ckucmd.c, 12 Mar 2010. + +From SMS for VMS: 'Added/documented P3 options INTSELECT, OLDFIB, OLDIP. +Disabled (commented out) automatic definition of NOSETTIME for VMS before +V7.2 (vms_ver .lts. "VMS_V72").' ckcdeb.h, ckcftp.c, ckcnet.c, ckuus[2567].c, +ckvfio.c, ckvker.com, ckvrtl.[ch], 15 Mar 2010. + +Exposed inesc[] and oldesc[] for VMS, so new INPUT command escape-sequence +stripping can work (really, chkaes() and related global variables should be +moved out of ck[uvd]con.c/ckucns.c and into a common module; do that later). +ckuusr.h, ckvcon.c, 15 Mar 2010. + +Built OK on Solaris9, Mac OS X 10.4.11, RHEL4 (32-bit), RHEL5 (64-bit), +AIX 5.3, SCO OpenServr 6.0.0... 15 Mar 2010. + +Not so good on VMS, turns out I made a typo in one of the VMS updates +(#ifndef OLDIP instead of #ifdef...). ckcnet.c, 16 Mar 2010. + +More from SMS for VMS, 16 Mar 2010: + . Set MAXPATH correctly for VMS, ckcdeb.h. + . NAM -> NAML, QIO replaces system( "SET PROTECTION"), bugfixes in + cvtdir() and nzltor(), ... (See comments): ckvfio.c, new ckvrms.h. + (The RMS code in ckvfio.c was almost totally rewritten) + . Moved "NAMX$*" (and related) macros to ckvrms.h, and renamed to + "NAMX_*" (and similar "$" -> "_"), moved "FIB_*" macros from ckvrtl.c. + +These changes are mainly to accommodate the ODS5 file system, which has +longer and mixed-case filenames, and also to execute certain commands +(e.g. for setting file protection, deleting directories) directly instead +of using a system() command. + +Built OK on VMS 8.3 (with and without network support). 16 Mar 2010. + +Failed to build on VMS 6.2. 16 Mar 2010. + +FreeBSD 8.0 has a hexdump() prototype that conflicts with the +hexdump macro defined in ckcdeb.h. Since the same thing is likely to happen +elsewhere, I changed the Kermit macro to ckhexdump as well all references to +it: ckcdeb.h, ckcftp.c, ckcnet.c, ckctel.c, ckuath.c, ckutio.c, 16 Mar 2010. + +Built OK on Digital Unix Tru-64 4.0E using "make osf", 16 Mar 2010. + +Tried again to build Digital Unix Tru64 4.0E using "make tru64-40e", but +something prevents it from picking up the termios symbols and it blows up in +ckutio.c, whereas this used to work in earlier C-Kermit versions. This is +the only Tru64 system I still have access to, so I can't tell if it's a +local peculiarity or what. Note that POSIX is not defined for this build. +But if I define it, I get into trouble with "struct timeval". Tried again +with "KFLAGS=-DPOSIX -DNOTIMEVAL" but that doesn't help. Tried "make +dec-osf" and that worked OK but oddly enough it makes a Kermit with less +features than "make osf". 16 Mar 2010. + +To go with MESSAGE and SET DEBUG MESSAGE, I added IF DEBUG, which is true +if SET DEBUG MESSAGE is not OFF and false otherwise. ckuusr.h, ckuus6.c, +16 Mar 2010. + +From SMS: Corrections to my merging of SMS's changes, ckcftp.c, ckvrtl.h. +Builds OK on VMS 6.2 now. Also did an SSL build on VMS 8.3 with OpenSSL +m0.9.7e and "OPENSSL_DISABLE_OLD_DES_SUPPORT" was included in P3 +automatically by Martin V's addition to ckvker.com. 17 Mar 2010. + +From SMS: #include earlier for VMS in ckcdeb.h to pick up off_t +before it is referenced. This allows C-Kermit to compile on VMS/Alpha 6.2 +but linking fails on fseeko() and ftello() (and yet, a functional executable +is created, and FSEEK works right). Builds the same way with no problems at +all on VMS 8.3 / Alpha. In this case we get the full 64-bit arithmetic... +Well, 62 bits: + + ATLAS::C-Kermit>( ^ 2 63) + 9223372036854775000.0 + ATLAS::C-Kermit>( ^ 2 62) + 4611686018427387904 + +whereas on VMS 6.2 we get integers only up to (^ 2 30). 17 Mar 2010. + +Changed the VMS build procedure to enable large file support automatically +for non-VAX and VMS 7.3 or greater. No reason not to include this feature. +Changed the sense of the F option to DISABLE large file support in the +unlikely case that C-Kermit is being built on a suitable platform but the +C library is older than VMS73_ACRTL-V0200, in which case fseeko() and +ftello() will come up missing at link time. ckvker.com, 18 Mar 2010. + +Changed VMS build procedure to include the FTP client in any network build +by default. Changed the sense of the I option to exclude the FTP client, +in case anybody would want to do that. ckvker.com, 18 Mar 2010. + +From SMS: updated dependencies in CKVKER.COM, fix the "don't reinclude me" +clause in CKVRTL.H. 19 Mar 2010. + +Built OK on VMS 6.2 and 8.3 with and without networking. Large file support +included automatically in VMS 8.3 FTP client included automatically in both +network builds. 19 Mar 2010. + +Changed hexdump() to ckhexdump() in ck_crp.c, which I missed before. +19 Mar 2010. + +---C-Kermit 9.0 Alpha.03--- + +In HP-UX with the bundled-non ANSI compiler, we get warnings about functions +such as endusershell(), which are declared void in the header files. But in +non-ANSI builds we defind VOID to be int rather than void, so our prototypes +are wrong. I checked that HP-UX 9, 10, and 11 all have void datatype and +changed the definition of VOID to void in those cases. ckcdeb.h, 29 Mar 2010. + +Fixed a typo in a debug() statement in cksplit() that was causing some +warnings. ckclib.c, 29 Mar 2010. + +Ditto in tls_load_certs(). ck_ssl.c, 29 Mar 2010. + +"make hpux1000o+ssl" files with: +/usr/ccs/bin/ld: Unsatisfied symbols: + __umoddi3 (code) + __udivdi3 (code) + __eprintf (code) + +It appears that OpenSSL (0.9.7c in this case) requires -lgcc. +And indeed hpux1000gcc+ssl builds fine. 29 Mar 2010. + +There are various warnings in the SSL code in ckutio.c, ckcftp.c, and +ckcnet.c about pointers not being assignment compatible, but I have learned +from experience not to try to fix these (see notes from 6 Oct 2009). +29 Mar 2010. + +connect(s, (struct sockaddr *)&hisctladdr, sizeof (hisctladdr)): In FTP, +this doesn't work on RHEL5 / Mac OX X 6.1/2 64-bit. But the connect() in +Telnet works. On Mac OS X 6.2 I tried changing the socket() call to be like +the one in ckcnet.c for Telnet, but it made no difference. On a RHEL5.4 +system on i386, FTP works fine, so it's not the Red Hat version. On Digital +Unix 4.0E 64-bit, same thing: + + 11:23:10.722 ftp_hookup[kermit.columbia.edu]=21 + 11:23:10.722 ftp hookup A[kermit.columbia.edu] + 11:23:10.722 ftp hookup C[kermit.columbia.edu] + 11:23:10.722 ftp hookup socket=4 + 11:23:10.722 ftp hookup HADDRLIST + 11:23:10.723 ftp hookup connect failed=13 + 11:23:10.723 ftp hookup bad + +13 = Permission denied: + + [EACCESS] Search permission is denied for a component of the path prefix; + or write access to the named socket is denied. + +On Gentoo Linux, also on Alpha, the errno is 51: Network is unreachable. +Clearly some data type in the sockets structs is out of whack. + +The third connect() argument is "address length". The address is a +struct sockaddr. About the third argument, RHEL5 "man connect" says: + + The third argument of connect() is in reality an int (and this is what + 4.x BSD and libc4 and libc5 have). Some POSIX confusion resulted in + the present socklen_t, also used by glibc. See also accept(2). + +Building on RHEL5 on x86_64, where size_t is 8 and socklen_t is 4, I get a +warning: + + ckcftp.c: In function 'ftp_hookup': + ckcftp.c:14667: warning: + comparison is always true due to limited range of data + +Referring to: + + if (hisctladdr.sin_addr.s_addr != (unsigned long) -1) + +This seems to be the problem; if I remove the (unsigned long) cast (in two +places), the problem goes away. Actually what I should be comparing it with +is INADDR_NONE, which is defined appropriately in some header file, e.g. as +0xffffffff. Also I define it explicitly as -1 if it is not defined in any +header file (as is the case in Solaris 9). Tested OK on 64-bit RHEL5, +32-bit RHEL5, Digital Unix 4.0E 64-bit, Solaris 9 32-bit, Mac OS X 10.4.11 +32-bit, Mac OS X 10.6.3 64-bit, AIX 5.3, Gentoo Linux 2.6.31 on Alpha +64-bit, NetBSD 5.0.1 32-bit.... ckcftp.c, 29 Mar 2010. + +---C-Kermit 9.0 Alpha.04--- + +Yesterday's VOID redefinition caused problems for HP-UX in ckuusx.c, in the +curses section where VOID is undef'd and not used to avoid a conflict with +curses.h. As a workaround I defined a new macro CKVOID with the same +definition as VOID and used it in the offending section of ckuusx. The real +solution is to replace all references to VOID with CKVOID (since VOID is +increasingly likely to cause conflicts), but a mass search and replace is +not without risks. ckcdeb.h, ckuusx.c, 30 Mar 2010. + +Changed VOID and CKVOID definition to be 'void' for all HP-UX (verified by +PeterE back to HP-UX 6.5, 1989). Still need to check this on HP-UX 5.21; +if that's an exception it can be done in the makefile. ckcdeb.h, 30 Mar 2010. + +The change I made to allow CONNECT to reestablish a previous SSH connection +prevented a new SSH connection to a different host to be made. Fixed in +ckuus7.c, 30 Mar 2010. + +Fixed mistaken extern declarations of krb4_errno and krb5_errno as strings +in nvlook(); they are ints. Built OK on Mac OS X 10.6.3. ckuus4.c, 30 Mar 2010. + +A fix to Trusted HP-UX makefile target from PeterE, to account for the +equivalence of +openssl and +ssl as target suffixes. 30 Mar 2010. + +Added a new function \fcvtcsets(string,cset1,cset1) that converts a string +from one character set to another. The csets are File Character-Set names. +ckuus4.c, 31 Mar 2010. + +Added a new function \fdecodehex(string,prefix) that decodes a string +containing prefixed hex bytes. Default prefix is %%, but any prefix of +one of two chars (such as % or 0x) can be specified. ckuusr.h, ckclib.h, +ckclib.c, ckuusr.c, 31 Mar 2010. + +Richard Nolde reports that Kermit can't find -lpam on Fedora 12 because it's +in /lib rather than /usr/lib. RHEL5 has symlinks, FC12 should too. Added a +note to the makefile. 1 Apr 2010. + +Build on Solaris 11 for the first time. Had to adjust ckuver.h to get the +version herald right. This was on a box that reported its architecture as +i86pc. 1 Apr 2010. + +Added MIME character-set names as invisible synonyms in the file and +terminal character-set tables, fcstab[] and tcstab[]. Note that not all the +character sets known to Kermit are registered in MIME. But at least now +MIME-registered character sets can be referred to by their MIME names, e.g. +ISO-8859-1, ISO646-ES, IBM437, WINDOWS-1252. These are not listed if you +type ? in a field that is parsing them, unless you type a letter first, +e.g. "i?" lists ISO- and IBM set names. Later maybe I'll make parallel +tables, or keyword attribute bit that says whether a name is MIME or not. +The real benefit of this change is that now Kermit can take its +character-set names from external sources like email headers or web logs. +ckuxla.c, 1 Apr 2010. + +Changed the IF command to accept a bare macro name its condition. This will +parse and execute correctly if the macro is defined and if it has a numeric +value, or if it is not defined, in which case it evaluates to 0 (FALSE). If +it is defined but has a non-numeric value, a parse error occurs. ckuus6.c, +2 Apr 2010. + +Added \fstringtype() function. Given a string argument, it tells whether +the string is 7bit, 8bit, utf8, binary, etc. ckuusr.h, ckuus[4x].c, +2 Apr 2010. + +Did a few builds to make sure there were no booboos. Solaris 9, NetBSD +5.01, Linux RHEL4, HP-UX 10.20 (non-ANSI compiler and ANSI optimizing +compiler), Mac OS X 10.4.11, SCO OSR 6.00. 5 Apr 2010. + +---C-Kermit 9.0 Alpha.05--- + +Increased maximum variable name length from 4K to 16K. Verified that +too-long names are caught and recovered from correctly. ckuusr.h, 6 Apr 2010. + +Implemented a new \fsplit() option for parsing CSV files, which turns out to +be a little complicated, because the separator is not just a comma, but a +comma and all its surrounding spaces. Also there are special quoting rules +for fields with embedded commas and fields with embedded quotes. ckclib.c, +7 Apr 2010. + +---C-Kermit 9.0 Alpha.06--- + +VMS changes from SMS. They build OK, Kermit file transfers are still OK, +but FTP text-mode GETs always hang on the 10th 8K network read. Couldn't +get a debug log this time. ckcmai.c, ckvfio.c, ckvrms.h, ckvker.com. +8 Apr 2010. + +Changing VNAML from 4K to 16K broke the build on HP-UX 9. Put it back to +4K. 9 Apr 2010. + +John Dunlap, running days-long stress tests between E-Kermit and C-Kermit, +found a bug in the packet-reading and -decoding code: If a NAK packet +arrives with its length field corrupted to indicate a bigger size, and there +are enough bytes following in the pipeline, ttinl() will return a too-long +packet (if there are not enough bytes waiting to be read, then ttinl() will +properly time out). In the bad case rpack() trusts the packet length, uses +it as the basis for computation of the block-check length, which is then +used to access memory that might not be there, causing (at least on John's +Linux system) a segmentation fault. John added the normal clause to check +the result of the block-check calculation, and I changed ttinl() to always +break on the eol character (normally carriage return), since this can never +appear in a packet, even if we "set control unprefix all". Also added a +check to ttinl() to protect against length fields corrupted into illegal +values. ckcfn2.c, ckutio.c, 13 Apr 2010. + +From Lewis McCarthy: + Based on code inspection, C-Kermit appears to have an SSL-related security + vulnerability analogous to that identified as CVE-2009-3767 (see e.g. + http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2009-3767). + + I'm attaching a patch for this issue relative to the revision of ck_ssl.c + obtained from a copy of http://www.columbia.edu/kermit/ftp/test/tar/x.zip + downloaded on 2010/07/30, which I believe is the latest. + + When this flaw was first widely publicized at last year's Black Hat + conference, it was claimed that some public certificate authorities had + indeed issued certificates that could be used to exploit this class of + vulnerability. As far as I know they have not revealed specifically which + public CA(s) had been found issuing such certificates. + Some references: http://www.mseclab.com/?p=180 + http://www.theregister.co.uk/2009/07/30/universal_ssl_certificate/ + +Patches added to ck_ssl.c, 4 Aug 2010. + +Peter Eichhorn reported that "RENAME ../x ." didn't work. This is a side +effect of the changes of 2006 to the RENAME command, there was a little +confusion in the renameone() routine; fixed in ckuus6.c, 4 Aug 2010. + +If only one file is FOPEN'd, FCLOSE given with no arguments would close it. +Turns out to be a bad idea. Example: program with an input and output file, +try to close the output file before it is opened by just typing FCLOSE; this +can mess up the input file. For safety FCLOSE has to require a channel +number or ALL. ckuus7.c, 4 Aug 2010. + +Added \fstrcmp(s1,s2,case,start,length), which has the advantage over IF +EQU,LGT,LLT that case sensitivity can be specified as a function arg, and +also substrings can be specified. ckuusr.h, ckuus[24].c, 5 Aug 2010. + +The CSV feature of Alpha.06 had a subtle flaw, namely that if the last item +in a comma separated list was enclosed within doublequotes with a trailing +space after the closing doublequote, a spurious empty final element would be +created in the result array. Fixed in cksplit(), ckclib.c, 5 Aug 2010. + +---Alpha.07--- + +The CSV feature of \fsplit() splits a comma-separated list into an array. +To turn the array back into a comma separated list, \fjoin(&a,\44,1) almost +works, except for elements contain literal doublequotes, such as: + + Mohammad "The Greatest" Ali + +This calls for making a symbolic CSV argument for \fjoin() like the one that +was made for \fsplit(): \fjoin(&a,CSV). Also \fjoin(&a,TSV) for +Tab-separated list. Thus if Kermit reads a record in CSV format, splits it +into an array, and then joins the array back into a CSV record, the result +will be equivalent to the original, according to the CSV definition. It +might not be identical, because if the result had extraneous spaces before +or after the separating commas, these are discarded, but that does not +affect the elements themselves. Furthermore it is now possible to convert +a comma-separated list into a tab-separated list, and vice versa (which is +not a simple matter of changing commas to tabs or vice versa). ckuus4.c, +12 Aug 2010. + +From Joop Boonen 26 Juli 2010: "Added HAVE_LOCKDEV as openSuSE >= 11.3 uses +lockdev but not baudboy. They use ttylock directly. The program code has +been added so the the program works without a problem." makefile, ckcdeb.h, +ckutio.c, ckuus5.c, 23 Aug 2010. + +---Alpha.08--- + +From Gary Mills at the U of Manitoba: convert Solaris version from BSD ptys +to streams ptys because there are only 48 BSD-style ptys and he was running +out. No code changes needed, the only change necessary was to add the +following flags to the makefile target: + + -DHAVE_STREAMS -DHAVE_GRANTPT -DHAVE_PTSNAME + -DPUSH_PTEM -DPUSH_LDTERM -DPUSH_TTCOMPAT + +makefile, ckcmai.c, 21 Sep 2010. + +Testing this in Solaris 9 I see that the DES library disappeared. Added +code to the solaris9 targets (also used by Solaris 10 and 11) to check for +this. makefile, 21 Sep 2010. + +The Solaris target checked the OpenSSL version automatically to set the +right flag, the Linux target didn't. Put the OpenSSL-version testing code +in the Linux target too. makefile, 21 Sep 2010. + +A couple minor changes to the tru64-51b makefile targets from Steven Schweda +but there still are some problems with the Tru64 Unix builds. +makefile, 21 Sep 2010. + +---Alpha.09--- + +\fcontents(\&a[3]) got an error if the array was declared but its dimension +was less than 3, which is bad when dealing with (say) an array created +dynamically by \fsplit(), which might or might not have a third element. +In case it doesn't -- i.e. in case we are referring to an out of range +element of any array that is declared -- we should just return a null +string, as we do with other types of variables that are not defined. +For that matter, ditto even if the array is not declared; what useful +purpose is served by throwing an error in this case? +ckuus4.c, 30 Dec 2010. + +cksplit() treats \ as a quoting character. If the source string contains +backslashes, they are swallowed (or, if doubled, one is kept). That's not +good for parsing external data, such as lines read from files, where there +are no quoting rules. This came up when parsing CSV files; as a workaround, +I made \fsplit() treat backslash as an ordinary character for CSV and TSV +splitting (a better solution might be yet another argument that specifies +a quote character). ckclib.c, 30 Dec 2010. + +Began converting C-Kermit to Open Source with the Simplified 3-Clause BSD +license. Updated the text for the INTRO, LICENSE, NEWS, and SUPPORT +commands. Fixed things so the copyright year to be displayed is defined in +one place (ck_cryear in ckcmai.c), rather than hardwired into text strings +all over the place. COPYING.TXT, ckcmai.c, ckuus[256].c, 2 Jun 2011. + +When I added MIME synonyms for Kermit character-set names, I left a bogus +entry in the tables ("windows-1251") that was in the wrong place +alphabetically, thus preventing most references to file character-set names +from working right. Removed the bogus entry. ckuxla.c, 2 Jun 2011. + +Most combinations work OK, but not translating Cyrillic text from UTF-8 +to Latin/Cyrillic, and probably the same would be true for any case of +converting from UTF-8 or UCS-2 to anything else. The problem was in +xgnbyte(), which converts the input stream from the specified character to +UCS2; it needed to make a special case for when the input file was already +Unicode. Believe it or not, this problem occurred at least as far back as +8.0.201 (9.5 years ago) and nobody noticed. So if the fix isn't perfect +probably nobody will notice that either. ckcfns.c, 3 Jun 2011. + +The SET BLOCK CHECK command did not parse all the items in its keyword +list. Fixed in ckuus3.c, 3 Jun 2011. + +For EM-APEX ocean floats project, where buoys in stormy waters have to +transmit data through an earth satellite using non-error-correcting modems, +John Dunlap ran exhaustive stress tests of Kermit protocol transfers through +a simulated connection that injected errors and delays and identified a +weakness in Kermit protocol when it is used under extremely bad conditions: +If a data byte of the S packet (or its Ack) is corrupted and the 1-byte +checksum is also corrupted in such a way that that the checksum matches the +corrupted data, the two Kermit programs will disagree as to the negotiated +parameters. For example, if file Sender's RPT field is changed from '~' to +'^', the receiver will decode the packet incorrectly. Ditto for most of the +other parameters. The result is that a corrupted file is received but +reported correct. John suggested a new mode of operation in which the Type +3 block check is used for all packets. Such a mode can not be negotiated +because the negotiation packet itself is assumed by all Kermit programs to +have a 1-byte checksum. Added SET BLOCK-CHECK 5 to the parser (with +invisible synonym FORCE-3". ckuus3.c, 3 Jun 2011. + +Added supporting code for SET BLOCK 5: ckcfn[23].c, ckcpro.w, ckcmai.c, +ckuus3.c, 3 Jun 2011. + +Added code to skip the heuristic that S and I packets always have block +check type 1. File transfer OK between two C-Kermits with SET BLOCK 5. +rpack(): ckcfn2.c, 5 Jun 2011. + +Made the file receiver put "5" in the block-check-type in its ACK to the +S-Packet. spar(): ckcfns.c, 5 Jun 2011. + +Now the question is: Can we make the file receiver automatically and safely +recognize a three-byte block check on an incoming S or I packet? It's +tricky because the block check field is not self-identified, it's just the +last "n" characters of string indicated by the length field, so correct +decoding of the packet depends on stateful knowledge of "n". How about this: +rpack() already knows what type of packet it is, so if it's an S or I packet +and the 8th byte of the data field is "5" and last 3 bytes, when interpreted +as the CRC, match the packet contents, then we accept the packet and switch +to BLOCK 5 mode. + +On the other hand, if the "5" was put there by corruption, the CRC should +catch the error. In that case we NAK the packet and presumably get a +different version back. There would be no reason to try to re-read the same +packet with a different block check, because the "5" could not possibly be +there legitimately unless it had a 3-byte CRC. To be clear, this is +cheating. We read the packet contents before we know the packet is correct, +then we check that it *is* correct. I made the 4-line change to rpack() +and it works OK in the absence of transmission errors. ckcfn2.c, 3 Jun 2011. + +So the various combinations should work as desired: + + . Sender and receiver both support and are told to SET BLOCK 5 ("SB5"). + . Sender SB5, but receiver doesn't support it (errors out). + . Sender SB5, receiver supports it but wasn't told (auto-recognizes it). + . Receiver SB5 but sender no (errors out). + +Note in the last case, the receiver should NOT automatically fall back to +standard behavior because if the user said SET BLOCK 5 that means every +packet MUST be protected by CRC to prevent the I/S packets from being +corrupted. + +Installed new HELP SET BLOCK-CHECK text. ckuus2.c, 5 Jun 2011. + +Autodownload didn't work when the S or I packet had a 3-byte block check +because kstart() checked it for a 1-byte checksum. Fixed in kstart(), +ckcfn2.c, 6 Jun 2011. However, older Kermit versions and programs that +claim to do "autodownload" will never recognize this type of packet. No +big deal since even if they did, the transfer would fail anyway. + +Added 'FORCE 3' to E-Kermit, called it EK 1.7. The option is "-b 5". Works +OK for sending and receiving, both with and without the new option. Also +works with "-b 5" if you send an S packet to it with '5' in the BCT field. +Changes were minimal, I have them all in ek17.diff. + +I could probably also make a new G-Kermit in about 10 minutes, but who cares +about G-Kermit... We already have two useful Kermit programs that +interoperate with the new protocol. 6 Jun 2011. + +Replaced the very inadequate help texts for functions \fword() and +\fsplit() with new ones. ckuus2.c, 6 Jun 2011. + +There were a couple reports of file corruption that I was saving for later. +Now that now is later I dug up the messages, files, and logs and it turns +out that nobody had reported a reproducible case of Kermit corrupting a +file. There have been non-reproducible cases though, almost certainly due +to corruption of the S or I packet or its ACK, which is why we now have SET +BLOCK 5. Even with BLOCK CHECK 5, there is no guarantee that the same thing +won't happen, it is just far less likely. Even if we added a 32-bit CRC or +even 64-bit one, there would still be a small chance it could happen. + +7 Jun 2011: + +Corrected various #ifdefs (or lack of them) when building C-Kermit with +different combinations of feature-selection options such as NOCSETS, NOICP, +NOLOCAL, NOSPL, NOUNICODE, etc. ckcfns.c ckcmai.c ckcxla.h ckuus2.c +ckuus4.c ckuus5.c ckuus6.c ckuusr.c, 7 Jun 2011. After running the script +that does all these builds (84 of them) I ran it again to make sure that +none of the changes broke builds that succeeded before the changes were made. + +Built OK on Solaris9 ("make solaris9") +Ditto with Krb5 and OpenSSL 0.9.8q ("make solaris9g+openssl+shadow+pam+zlib") + +Built OK on Mac OS X 10.4.11 ("make macosx"). +Also "make macosx+krb5+openssl. + +Built OK on Linux RHEL4 ("make linux"). +Built OK on Linux RHEL4 with OpenSSL 0.9.7a ("make linux+ssl"). +Built OK on Linux RHEL5 ("make linux"). + +"make linux+ssl" fails on RHEL5 because of DES, even though the target +tests for the presence or absence of the DES libraries. In this case the +libraries are there but they lack the functions des_ecb3_encrypt, +des_random_seed, and des_set_odd_parity. The build succeeds as: + + make linux+ssl KFLAGS=-UCK_SSL + +Since DES is now considered harmful, Jeff Altman suggests that all OpenSSL +builds, even for old versions, should omit it ("If you are building with +openssl and no kerberos or srp, just disable DES. Disabling DES will impact +telnet and rlogin but it won't matter if you have no ability to negotiate a +session key"). + +From Ian Beckwith, patches for Debian Linux: + . Change all '-' to '\(hy' in man page (new pedantry): ckuker.nr. + . Make IKSD authentication (using PAM) ask for a password when an invalid + username has been given, to avoid disclosing which account names are valid: + ckufio.c, ckuus7.c. + . Fix spelling errors: ckcftp.c, ckuus2.c, ckuker.nr, ckcpro.w, ckuusr.h. + . Patch makefile to support install to a staging area with DESTDIR. + . Some other patches (mainly for typos) were for plain-text documentation + files that were generated from Web pages; I updated the web pages. + +A big corporate C-Kermit user has an application where a local C-Kermit +makes a connection to a remote one, uploads some files, and then if the +server has any new patch files for the local, it sends the patches and +does a REMOTE HOST command to run the patch program. This stopped working +in C-Kermit 6.0 or 7.0 when I put a check to prevent it, because "it makes +no sense to send REMOTE commands to the local end, because the results are +sent back to the remote to be displayed on its screen but it has no screen". +That may be true, but if the user needs to control the local from the +remote, they should be able to. I removed the checks. This doesn't solve +the problem of where the output goes; ideally it would go to the local +screen but I don't see any elegant and simple way to make that change. +However the output redirectors can still be used with the REMOTE command +so the results can be captured to a remote file, which could then be sent. +ckuus7.c, 7 Jun 2011. + +Changed SET VARIABLE-EVALUATION to SET COMMAND VARIABLE-EVALUATION, but left +the former version available. ckuusr.c, 9 Jun 2011. + +Documented the SET COMMAND VARIABLE-EVALUATION command, which I added in +2008. ck90.html, 9 Jun 2011. + +Renamed all old Mac OS X makefile targets to have the prefix "old" to avoid +confusing them with the current targets, and made macosx10 a synonym for +macosx, so those who used previous makefiles will get a current target +without having to know the new name. makefile, 9 Jun 2011. + +Added XMESSAGE, which is to MESSAGE as XECHO is ECHO: prints the text +without a line terminator, so it can be continued by subsequent [X]MESSAGE +commands. ckuusr.[ch], 9 Jun 2011. + +Back to "make linux+ssl" on RHEL5... I took the coward's way out and added +code to the makefile target to check whether the build worked (somebody let +me know if there is a better way to check), and if not to give a message +suggesting they "make clean ; make linux+ssl KFLAGS=-UCK_DES". makefile, +9 Jun 2011. + +Noticed that \frecurse() would dump core if called with no arguments. +Fixed in ckuus4.c, 9 Jun 2011. + +Added \q() as an alternative to the more verbose \fliteral() for quoting +strings that contain characters (like \) that would otherwise be significant +to Kermit. It's more efficient because it isn't a function call, and 'q' +is an intuitive letter to mean 'quote'. It also works better than +\fliteral() because functions treat commas and braces specially. ckuus4.c, +10 Jun 2011. + +Built OK on VMS 8.3 on Alpha, no net. DEC C caught a couple glitches in the +new code that gcc didn't catch, which I fixed. ckuus[25].c, 10 Jun 2011. + +Built OK on VMS 8.3 on Alpha with Multinet 5.3. The SSL build failed but +I'm not going to worry about it. 10 Jun 2011. + +Built OK on NetBSD 5.1. 10 Jun 2011. + +Tried to resurrect my old "build-all" machine, an IBM Netfinity 3500 from +1997 with 20-some mountable bootable hard disks with lots of 1990s OS's on +them. No dice. I can see the BIOS but not the hard disks. The +configuration is still correct because it tries to boot from the mountable +hard disk, but it fails (I tried six different ones). + +Tried to resurrect my old Siemens Nixdorf RM 200 MIPS machine. Booted OK, +headless even, but makes a hellish high-pitched whine, like a dentist drill. +It's pretty slow too. "make sinix542" (for SINIX 5.4.2) bombed at link +time on no rdchk(). Fixed by #including . ckutio.c, 10 Jun 2011. + +Tried to resurrect my old SCO Xenix 2.3.4 machine, also headless. Amazingly +it still works; it can't use a monitor but I can Telnet to it. Had to tweak +some #ifdefs but I got a no-net version built successfully. According to my +notes, it hasn't been possible to build with TCP/IP since C-Kermit 8.0, +but how many people ever had SCO Xenix 2.3.4 with TCP/IP anyway? Anyway we +still have the binaries for C-Kermit 7.0. ckuus4.c, 10 Jun 2011. + +Built OK on AIX 5.3. Built OK on Solaris 10. 11 Jun 2011. + +Tried harder to revive the build-all machine, now it sort of works, but not +all of the bootable OS's work. Built C-Kermit 9.0 OK on OpenBSD 3.0. Built +OK on QNX 4.25 but had to #ifdef references to IXANY in ckutio.c and ckupty. +Built OK on NetBSD 1.5.1 (2000). Tried "make netbsd+ssl" on this one, it's +OpenSSL 0.9.5a 1 Apr 2000, but it bombs out in ckuath.c, no big deal. +Another problem in NetBSD 1.5.2 is that even though off_t is 8, CK_OFF_T +is 4. Worth noting but not worth fixing unless someone else notices. +13 Jun 2011. + +SuSE 7.0... boots OK but telnet server doesn't work. Can telnet out but +it's too flaky, connection drops if I try to transfer a file. + +OpenBSD 2.5 [1999] OK. Red Hat 7.1 OK. Red Hat 7.1 with OpenSSL 0.9.6 +not OK, same error as with 0.9.5a: + +ckuath.c +In file included from ck_ssl.h:48, + from ckuath.c:225: +/usr/include/openssl/des.h:77: warning: redefinition of `Block' +ckuat2.h:86: warning: `Block' previously declared here +/usr/include/openssl/des.h:83: redefinition of `struct des_ks_struct' +/usr/include/openssl/des.h:91: warning: redefinition of `Schedule' +ckuat2.h:90: warning: `Schedule' previously declared here + +So it appears that OpenSSL support is broken for pre-0.9.7. Tried +building it again with -UCK_SSL (since the errors are originating from +from des.h)... But it still failed exactly the same way. I found +#includes for des.h in ckuath.c and and ck_ssl.h and #ifdef'd them out, +but it still fails: + +In file included from /usr/include/openssl/evp.h:89, + from /usr/include/openssl/x509.h:67, + from /usr/include/openssl/ssl.h:69, + from ck_ssl.h:51, + from ckuath.c:227: +/usr/include/openssl/des.h:77: warning: redefinition of `Block' +ckuat2.h:86: warning: `Block' previously declared here +/usr/include/openssl/des.h:83: redefinition of `struct des_ks_struct' +/usr/include/openssl/des.h:91: warning: redefinition of `Schedule' +ckuat2.h:90: warning: `Schedule' previously declared here + +Built OK on Debian 2.1. 13 Jun 2011. + +On FreeBSD 4.4, it blows up with: +ckufio.c: In function vpass': +ckufio.c:8201: conflicting types for 'initgroups' +/usr/include/unistd.h:154: previous declaration of 'initgroups' +ckufio.c:8201: warning: extern declaration of 'initgroups' doesn't match global +one. Fixed by defining NODCLINITGROUPS for FreeBSD in ckufio.c. It might not +be the right fix, but I don't have a lot of other FreeBSD versions to +compare with. Anyway now it builds OK on 4.4, and also on FreeBSD 3.3. +ckufio.c, 13 Jun 2011. + +Tried to build on SCO Open Server 5.0.7 but it fails at link time because +it can't find rdchk(). But it's supposed to be there! Come back to this +later... + +Red Hat 6.1 i386 32/64 linux 2332545 +Red Hat 7.1 i386 32/64 linux 2368528 +Red Hat EL4 i386 32/74 linux 2363067 +Red Hat EL5.6 i386 64 linux 2371279 +Solaris9 sparc 32/64 solaris9 2849896 +Solaris9+ssl sparc 32/64 solaris9 5021764 +Solaris10 sparc 32/64 solaris10 2855776 +QNX i386 32 qnx32 2012323 +NetBSD 1.5.1 i386 32/64 netbsd 2198055 +NetBSD 5.1 i386 32/64 netbsd 2159863 +OpenBSD 2.5 i386 32/64 openbsd 2236036 +Mac OS X 10.6.7 x86_64 64 macosx 2.7M +Mac OS X 10.4.11 ppc 32/64 macosx 2496304 +Debian 2.1 i386 32/64 linux 2213221 +FreeBSD 4.4 i386 32/64 freebsd 2291333 +FreeBSD 3.3 i386 32/64 freebsd 2147370 +SINIX 5.42 mips 32 sinix542 3319325 (1995) +SCO Unixware 2.1.3 i386 32 uw213 2242176 +SCO OSR6.0.0 i386 32/64 sco_osr600 2368300 + +More builds, 14 June 2011: + +VMS 6.2 alpha 32 make mn 2556928 No TCP/IP +VMS 6.2 alpha 32 make m 3112960 UCX 4.0 +Solaris 11 i386 32/64 solaris11 2823860 +Solaris 11 i386 32/64 solaris11+ssl 2993660 OpenSSL 0.9.8l +NetBSD 5.1 i386 32/64 netbsd+krb5 2307855 Kerberos 5 +Linux Slackware 12.1.0 i386 32/65 linux 2175754 +Linux Fedora 14 i386 32/64 linux 2256514 +Linux Fedora 14 i386 32/64 linux+ssl ....... OpenSSL 1.0.0d +Linux Fedora 14 i386 32/64 linux+krb 2449614 (*) + +(*) make linux+krb5 "LIBS=$LIBS /lib/libk5crypto.so.3 /lib/libcom_err.so.2" + +Noticed that netbsd+ssl build on NetBSD 5.1 said "NetBSD 1.5" in its banner. +Fixed by replacing the old hardwired target with the new "subroutinized" +target a'la linux+ssl and adapting it to NetBSD. makefile, 15 Jun 2011. + +Same deal for Kerberos 5, make a new netbsd+krb5 target and it builds ok, +at least once one figures out where the Kerberos headers and libs are. +makefile, 15 Jun 2011. + +Same deal for the netbsdnc target, now it simply defined NOCURSES and +chains to the main netbsd target. makefile, 15 Jun 2011. + +Tried to build with Kerberos 5 on Solaris, fails because the DES library +no longer exists. This one is beyond me, sorry. + +Made new targets for MirBSD, mirbsd and mirbsd+ssl, makefile 15 Jun 2011. + +In OpenSUSE 11.2 with OpenSSL 0.9.8r we bomb on undefined references from +various DES library routines. Builds OK without DES. + +Various linux+krb5 builds fail because can't find -lgssapi_krb5 + +SSL builds with OpenSSL < 0.9.7 fail even though there is code to support +the older SSL. + +Fixed some printf %ld vs int instances in the sizeofs section of SHOW FEATURES. +ckuus5.c, 15 Jun 2011. + +Fixed the new linux+ssl target to actually use the SSLINC and SSLLIBS +definitions, oops. makefile, 15 Jun 2011. + +15 June 2011 builds (Beta.01): + +AIX 5.3 ppc 32/64 aix+ssl 3283846 OpenSSL 0.9.8m +NetBSD 5.1 i386 32/64 netbsd 2159863 +NetBSD 5.1 i386 32/64 netbsd+ssl 2350274 OpenSSL 0.9.9-dev +NetBSD 5.1 i386 32/64 netbsd+krb5 2349627 MIT Krb5 1.6.3 +FreeBSD 8.2 i386 32/64 freebsd 2298414 +FreeBSD 8.2 i386 32/64 freebsd+ssl 2448961 OpenSSL 0.9.8q +OpenBSD 4.7 i386 32/64 openbsd 2266132 +OpenBSD 4.7 i386 32/64 openbsd+ssl 2409263 OpenSSL 0.9.8k +MirBSD 10 i386 32/64 mirbsd 2216601 +MirBSD 10 i386 32/64 mirbsd+ssl 2358318 OpenSSL 0.9.8r +OpenSuse 11.2 x86_64 64 linux 2348468 +OpenSuse 11.2 x86_64 64 linux+ssl (*) 2546540 OpenSSL 0.9.8r +RHEL 5.6 ia64 64 linux 4390687 +RHEL 5.6 ia64 64 linux+ssl (*) 4775007 OpenSSL 0.9.8e +Ubuntu 9.10 i386 32/64 linux 2275523 +Ubuntu 9.10 i386 32/64 linux+ssl 2466708 OpenSSL 0.9.8r +Gentoo 1.12.13 ppc 32/64 linux 2386597 +Gentoo 1.12.13 ppc64 64 linux 2749015 +Gentoo 1.12.13 ppc64 64 linux+ssl 3002150 OpenSSL 0.9.8r +Gentoo 1.12.13 sparc 32/64 linux 2478382 +Gentoo 1.12.13 sparc 32/64 linux+ssl 2690499 OpenSSL 0.9.8r +Solaris 9 sparc 32/64 solaris9 2849896 +Solaris 10 i386 32/64 solaris10 2837620 +IRIX 6.5 R10000 32/64 irix65 2869704 + +* and KFLAGS=-UCK_DES + +Tried building on NetBSD 5.1 with Heimdal Kerberos using: + +make netbsd+krb5 \ + "KFLAGS=-DHEIMDAL" \ + "K5INC=-I/usr/include" \ + "K5LIB=-L/usr/lib" + +It found all its headers OK, but it blew up in ckuath.c. Small wonder, +ckccfg.html says: + +HEIMDAL + Should be defined if Kerberos V support is provided by HEIMDAL. Support + for this option is not complete in C-Kermit 8.0. Anyone interested in + working on this should contact kermit-support. + +'krb5-config --version' gives the MIT Kerberos 5 version number. + +Make a new netbsd+krb5+ssl target based on the combination of the new +netbsd+ssl and netbsd+krb5 targets. There were lots of warnings in the +compilation but no errors, but it produced an executable that starts and +does normal things but I have no idea if the SSL or Kerberos functions work. +makefile, 16 Jun 2011. + +Changed the cu-solaris9-krb5 target to test for the presence of DES because +DES isn't there, to see if this would allow a Kerberos build to proceed. +And it worked, amazing. At least the build completed, I have no way to test +the Kerberos part. makefile, 16 Jun 2011. + +Updated the solaris9+ssl target to do the DES testing. makefile, 16 Jun 2011. + +Updated cu-solaris+krb5 target to test whether the GSSAPI library is called +libgassapi or libgassapi_krb5. makefile, 16 Jun 2011. + +Added lots of tests to the Linux Kerberos 5 entries, linux+krb5 and +linux+krb5+ssl, because some have libk5crypto and some don't; some have +libcom_err and some don't; and some have libgssapi_krb5 (e.g. RHEL5, +OpenSuse 11.2) whereas others have libgssapi (Gentoo). + +16 June 2011 builds (Beta.01): + +NetBSD 5.1 i386 32/64 netbsd+krb5+ssl 2451757 OpenSSL 0.9.9 MIT Krb5 1.6.3 +Solaris 9 sparc 32/64 solaris9+krb5 2543036 MIT Kerberos 5 1.7.1 +Solaris 9 sparc 32/64 solaris9+ssl 5021544 OpenSSL 0.9.8q (gcc) +Gentoo... ppc 32/64 linux 2386597 +Gentoo... ppc 32/64 linux+ssl 2593561 OpenSSL 0.9.8r +Gentoo... ppc64 64 linux 2749015 +Gentoo... ppc64 64 linux+ssl 3002150 OpenSSL 0.9.8r +RHEL5 x86_64 64 linux+krb5 (*) 2563878 MIT Kerberos 5 1.6.1 +RHEL5 x86_64 64 linux+krb5+ssl(*) 2563878 MIT Kerberos 5 1.6.1 +Fedora 14 i386 32/64 linux+krb5+ssl 2539891 MIT Krb5 + OpenSSL 0.9.8r + +* KFLAGS=-UCK_DES + +--- C-Kermit 9.0.299 Beta.01 --- + +sizeof() can return a long or an int, so neither printf("%d",sizeof(blah)); +or printf("%ld",sizeof(blah)); can be used everywhere. Changed the +"sizeofs" section of SHOW FEATURES in the dumbest (and therefore most +portable) way to squelch the warnings. ckuus5.c, 17 Jun 2011. + +From John Dunlap: "Watching the server screen led me to offer a cosmetic +patch for ckuusx.c. I noticed that the server screen said it was +"RESENDING" when it really wasn't. The attached patch emits blanks to +insure that old labels are completely erased." ckuusx.c, 17 Jun 2011. + +Nelson Beebe found two places where I had SSLLIBS in the makefile instead of +SSLLIB. makefile, 18 Jun 2011. + +More important he knew how to force gcc to load the right header files for +OpenSSL 1.0.0d (by using '-isystem' rather than '-I'). Previously it was +using the 0.9.8r header files but linking with the 1.0.0d libraries. This +is not in the sources or makefile; it's done when giving the 'make' command: + + export PATH=/usr/bin:$PATH + export SSLINC=-isystem/usr/include + export "SSLLIB=-L/usr/lib -Wl,-rpath,/usr/lib" + make linux+ssl + +Folded the previous linux+openssl+zlib+shadow+pam and linux+openssl+shadow +targets into linux+ssl. Checked the linuxso (scripting only) target, builds +OK, 600K. Made new subroutinized linux+krb5+krb4 target but can't find +anyplace to test it. Made new subroutinized linux+shadow+pam target, works +fine on RHEL4. Revised comments and lists again. makefile, 18 Jun 2011. + +For the pluggable-disk OS's that boot OK but lack a working network, I +rigged up a serial connection using a DB9-FF null modem cable, and then a +DB9-MF modem cable to make it reach. I don't see any modem signals on +either end, but the data goes through OK. COM1 on the desktop PC, +/dev/ttyS1 or whatever on Lab. Since there are no modem signals, can't use +RTS/CTS. At 57600bps with Xon/Xoff, 500-byte packets and sliding windows, +transfers work OK at about 5000cps using 5 window slots; takes 8 minutes to +transfer the gzipped C-Kermit tarball. Kermit to the rescue. 19 Jun 2011. + +Transferred the tarball over serial ports to SCO OSR5.0.5 at 38.4Kbps, the +highest speed supported, 12 minutes, no errors, 3300cps. Unpack, make +sco32v505udk, OK. Also built the TCP/IP version and it almost made an +outbound connection, but only once (not a Kermit problem but something with +the TCP/IP stack). 19 Jun 2011. + +Ditto for Solaris 2.6/i386, except 57.6Kbps, 4K-byte packets, no problem. +Solaris 8/i386, ditto. 19 Jun 2011. + +SCO OpenServer 5.0.5 i386 32 sco32v505udk 1940964 No TCP/IP +SCO OpenServer 5.0.5 i386 32 sco32v505udknet 2314668 With TCP/IP +Sun Solaris 2.6 i386 32 solaris26g 4661368 +Sun Solaris 8 i386 32 solaris8g 4675432 + +When using compact substring notation, \s(xx[4]) returns the whole string +xx starting at position 4, but \s(xx[4:]) returns an empty string. Fixed +the latter to be like the former. ckuus5.c, 20 Jun 2010. + +Really it would have been nicer if \s(xx[4]) returned a single character, +the 4th character of xx, but it's too late now. Added another "separator" +character '.' (period) for that: \s(xx[4.]) is the 4th character of xx. +ckuus4.c, 20 Jun 2010. + +Back to SCO OSR5.0.7... This failed before because 'rdchk' came up unknown +at link time, unlike all previous OSR5's, that used rdchk() in place of the +FIONREAD ioctl. Added #ifdefs to make a special case for 5.0.7. I'm not +sure this is the best way, but this is the minimal change to get it to work. +If anybody cares, maybe the same can be done for previous OSR5 releases. +ckutio.c, 20 Jun 2010 (search for SCO_OSR507). + +SCO OpenServer 5.0.7 i386 32 sco32v507 1895724 No TCP/IP +SCO OpenServer 5.0.7 i386 32 sco32v507net 2246792 With TCP/IP + +Checked current code on RHEL4, found that my GSSAPI-lib finding makefile +target didn't look in enough places; added some more. makefile, 21 Jun 2011. + +Got reports back on HPUX from Peter Eichhorn, almost all good on HP-UX 7, 8, +9, 10, and 11. 21-22 Jun 2011. + +Got access to Debian 5.0 and 7-to-be ("Wheezy/Sid"). Regular 'make linux' is +OK in Debian 5, but in 7 can't find crypt, res_search, or dn_expand; had +to add more library search clauses to 'make linux'. makefile, 21 Jun 2011. + +In Debian 7.0, libk5crypto could not be found without adding another clause +to 'make linux+krb5'. That done, the SSL build (1.0.0d) was OK, as well as +the krb5+ssl one. makefile, 21 Jun 2011. + +I found a Linux box that had both Kerberos 4 and 5 installed and tried 'make +linux+krb5+krb4', which failed because of missing DES functions. Tried +'make linux+krb5+krb4 KFLAGS=-UCK_DES', but that fails too, even though it +doesn't fail for Kerberos 5 alone, so probably some Krb4 code is making +unguarded calls to the DES routines. What is really needed is a way to +completely strip all DES references from any given build, code and makefile, +a big deal. 21 Jun 2011. + +Fixed some typos in COPYING.TXT (noticed by Ian Beckwith). 24 Jun 2011. + +Got access to perhaps the last living 4.3BSD VAX system. It doesn't have +SEEK_CUR so I had to #ifdef out the \fpicture() function. Aside from that, +no problems. ckuus4.c, 24 Jun 2011. + +I had been wanting the S-Expression (ROUND x) to allow a second argument n, +which, if given, tells where the rounding should occur. If n is positive, +the number is rounded to n decimal places. If zero, it is rounded to the +nearest integet. If positive, the number is rounded to the nearest power of +10; e.g. -2 means "to the nearest hundred". If ROUND is used as before, +with one argument, it works as before. ckclib.c, ckuus3.c, 25 Jun 2011. + +From Arthur Marsh, a few more directories to test for libresolv in Linux. +makefile, 26 Jun 2011. + +From Martin Vorlaender, a fix for the VMS file-transfer display and +statistics, a place where a file length wasn't being cast to CK_OFF_T +in zchki(). ckvfio.c, 28 Jun 2011. + +Updated version to 9.0.300 and removed the Beta designation. +ckcmai.c, makefile, 28 Jun 2011. + +Removed solaris9_64 target from makefile. It builds but it doesn't work +at all. 30 Jun 2011. + +--- C-Kermit 9.0.300 --- + +On Solaris 10 and 11, DNS lookups don't work. It seems these Solaris +versions have INADDRX and INADDRX_NONE defined, thus triggering the code in +ckcnet.c, ckucns.c, and ckcftp.c #ifdef'd on these symbols, but that code +doesn't work in this case. This happens building with gcc as well as with +Sun cc. Put #ifdefs in ckcnet.h to undefine these symbols (if they are +defined after including all the header files) for Solaris. I didn't bother +trying to differentiate the Solaris versions because the symbols are not +defined in Solaris 9 or earlier, and they should not be used in Solaris 10 +or 11. ckcnet.h, 6 July 2011. + +From SMS: To avoid the %CC-W-PTRMISMATCH1 complaints from ck_ssl.c, add +two (harmless) type casts at lines 2460 and 2773, 6 July 2011. + +Built and tested on Solaris 9, Solaris 10, and RHEL5. 6 July 2011. + +--- C-Kermit 9.0.301 Beta.01 --- + +Updated version text and date. ckcmai.c, makefile, 11 July 2011. + +--- C-Kermit 9.0.301 --- + +After the initial release I made some small changes that affect only HP-UX +5.x: added -DVOID=void and -DCKVOID=void to the hpux0500 makefile targets, +and put #ifdefs around #include , which (in the WinTCP case) didn't +protect itself against multiple inclusion (which is happening in other +header files, not in Kermit). makefile, ckucmd.c, ckucon.c, ckutio.c, +ckufio.c, ckcnet.c, ckcftp.c, 14 July 2011. + +In the new copyright notice, copied from the BSD license template, +one instance of "the " was not replaced by "Columbia +University". Fixed in ckcmai.c, 19 July 2011. + +Added another search term for lk5crypto in the linux+krb5 targets. +makefile, 20 July 2011. + +Added and successfully used a new solaris9+krb5+ssl target. +makefile, 8 Aug 2011. + +FreeBSD 9 switched from utmp to utmpx, which broke compilation of Kermit on +that version. Furthermore, the UUCP lockfile convensions changed in FreeBSD +8, which did not prevent C-Kermit from compiling, but any attempt to lock a +terminal or pty device would fail. Thanks to Alexey Dokuchaev "danfe" for +finding and patching the problems. I undid the patches and fixed the code +so it didn't need to be patched, except for some declarations in the +ck_crp.c module, which I felt had better not be changed without thoroughly +testing the changes on dozens of different platforms, which I don't have +time to do (in any case, it builds OK on FreeBSD 9 without the patch). In +particular I made completely new makefile targets for FreeBSD 4.0 and later, +which automatically detect FreeBSD 8 and FreeBSD 9 to enable the appropriate +feature tests in the code, for a regular build and a build with OpenSSL. +These changes should affect only FreeBSD. ckutio.c, ckufio.c, ck_crp.c, +ckuus5.c, makefile, 20 Aug 2011. + +Changed the version to 9.0.302, ckcmai.c, 20 Aug 2011. + +Fixed freebsd+ssl and netbsd+ssl, netbsd+krb5, and netbsd+krb5+ssl targets +to have CC=$(CC) instead of CC=gcc; ditto for CC2, and adjusted line breaks +in freebsd and freebsd+ssl targets. makefile, 21 Aug 2011. + +--- 9.0.302 --- + +--------------------------------- +***************************