2 [1]The Columbia Crown The Kermit Project | Columbia University
3 612 West 115th Street, New York NY 10025 USA o [2]kermit@columbia.edu
5 [3]Home [4]Kermit 95 [5]C-Kermit [6]Scripts [7]Current [8]New [9]FAQ
9 [11]CLICK HERE to read about some of these items.
11 [12]Table of platforms [13]Book: Using C-Kermit [14]Download
14 C-Kermit 9.0 Update Notes
16 Note: C-Kermit 9.0.301 contains a correction that applies only to
18 C-Kermit 9.0.302 contains corrections that apply only to FreeBSD 8
21 * [16]How to Test Large-File Transfer
22 * [17]Arithmetic with Large Integers
23 * [18]FORCE-3 Packet Protocol
24 * [19]Variable Evaluation
26 * [20]The RENAME Command You Always Wanted
27 * [21]Other New Features
28 * [22]Incompatibilities
29 * [23]What's Not In C-Kermit 9.0
32 * [25]Demonstration: Secure POP mail fetcher
33 * [26]Demonstration: HP Switch Configuration Backup
34 * [27]Demonstration: HP iLO Blade Configuration
35 * [28]Demonstration: IBM/Rolm/Siemens CBX Management
36 * [29]Demonstration: CSV and TSV Files
37 * [30]Demonstration Scripts for Webmasters
39 This is the third supplement to [31]Using C-Kermit, Second Edition. I
40 apologize for the scattered nature of the information and I hope I can
41 organize it and gather it all into one place for easy and definitive
42 reference some day. It's a big job so it depends on the demand. For the
43 time being the definitive reference and introduction is the book (which
44 is now available also in a [32]Kindle Edition), plus the [33]C-Kermit
45 7.0 update, [34]C-Kermit 8.0 update, and now this one. Plus tons of
46 other web pages on this site, sample script programs, and so on.
48 In version 6.0, C-Kermit was a pretty powerful and flexible
49 communication program with scripting capabilities. By version 9.0, I'd
50 like to think of it more as a scripting language with built-in
51 communications. You can get an idea of the kinds of programs you can
52 write in Kermit language [35]here. You can develop programs quickly
53 because it's an interactive program, not a compiler. The scripting
54 language is the command language. Kind of like the Unix shell but
55 "somewhat" less cryptic, including concepts not only from C but from
56 PL/I, Snobol, LISP, Bliss, and Smalltalk. The language itself is built
57 upon the command language of the much-loved [36]DECSYSTEM-20 from the
58 1970s and 80s, the Clipper Ship of the Text Era. (Text is not a bad
59 word. Those of us who can touch-type and who are proficient in
60 text-based computing environments like Unix shell or VMS DCL are likely
61 to be orders of magnitude more productive than users of GUIs.)
63 Thanks to (at least) Jeff Altman, William Bader, Ian Beckwith, Nelson
64 Beebe, Gerry Belanger, Joop Boonen, Rob Brown, Christian Corti, Alexey
65 Dokuchaev, John Dunlap, Peter Eichhorn, Carl Friedberg, Terry Kennedy,
66 Günter Knauf, Jason Lehr, Arthur Marsh, Lewis McCarthy, Gary Mills, Ed
67 Ravin, Jonathan Reams, Mike Rechtman, Mark Sapiro, Steven Schweda
68 (SMS), Kinjal Shah, Michael Sokolov, Andy Tanenbaum, Seth Theriault,
69 Zach A. Thomas, Martin Vorländer, and Eric Weaver for assistance, and
70 to Hewlett-Packard Company for support.
72 - Frank da Cruz [37]fdc@columbia.edu, 30 June 2011
74 P.S. It occurred to me just before the end of the day that maybe I
75 should back up the Kermit website on DVD, just in case. Using
76 [38]Kermit 95 on the desktop over an SSH connection to the Unix file
77 system where the website resides, I made a fresh directory on the PC,
78 CD'd to it, and on Unix cd'd to the Website directory, and told
81 C-Kermit> send /recursive /dotfiles /nobackup *
83 and it re-created the website directory tree in the PC directory, text
84 files correctly converted to Windows format and binary files correctly
85 left as-is. The /dotfiles switch means to include files such as
86 .htaccess whose names start with a dot (period), and the /nobackup
87 switch means to skip backup files created by EMACs (such as
88 index.html.~243~). And then I did the same with the FTP sites, about
89 8GB in all. Watching the file-transfer display was kind of like having
90 30 years of my life flash before my eyes in a few minutes. Then I
91 copied the two directories to DVD (the FTP site had to be split over 2
92 DVDs). The whole operation took under half an hour. The directory tree
93 on the CD is directly usable in Windows, Unix, or any other operating
94 system (unlike if I had transferred the files all in binary mode or all
95 in text mode, or if I had made, say, a gzipped tar archive or a zip
96 archive). I believe that, to this day, Kermit is the only software that
97 can do this. If someday I have to upload from these DVDs to Unix, VMS,
98 or any other operating system, it can be done exactly the same way,
99 with any necessary conversions on text files done automatically, and
100 binary files left intact, recursively through a whole very large
103 What's New in General
105 Very briefly, the major items:
106 * [39]Open Source license.
107 * [40]64-bit file access and transfer and 64-bit integer arithmetic
108 on most common platforms.
109 * Support for recent releases of Linux, Mac OS X, *BSD, etc ([41]see
111 * Support for newer OpenSSL releases up to and including 1.0.0d
113 * [43]Strengthened error checking for file transfer under extremely
115 * [44]Simplified semantics for variables used in scripts.
116 * Super-handy [45]extensions to the RENAME command.
117 * Other scripting improvements including support for reading and
118 writing [46]CSV and TSV files.
119 * [47]MIME character-set names are now recognized.
120 * Improved logging and debugging (see demo [48]here).
121 * Lots more described or listed below, and [49]here.
125 C-Kermit 9.0 has the [50]Revised 3-Clause BSD License, an open source
126 license approved by OSI, the [51]Open Source Initiative.
130 Kermit is, first and foremost, a file-transfer program. One might
131 expect it to be able to transfer any kind of file, but that has been
132 decreasingly the case as file sizes began to cross the 2 gigabyte
135 The biggest change since C-Kermit 8.0.211 is support for large files on
136 platforms that support them. A "large file" is one whose size is
137 greater than 2^31-1 (2,147,483,647) bytes (2GB-1); that is, one whose
138 size requires more than 31 bits to represent. Before now, Kermit was
139 able to access such files only on 100% 64-bit platforms such as Digital
140 Unix, later known as Tru64 Unix. In the new release, Kermit takes
141 advantage of the X/Open Single UNIX Specification Version 2 (UNIX 98)
142 Large File Support (LFS) specification, which allows 32-bit platforms
143 to create, access, and manage files larger than 2GB.
145 Accommodating large files required code changes in many modules,
146 affecting not only file transfer, but also file management functions
147 from directory listings to local file manipulation, plus the user
148 interface itself to allow entry and display of large numbers. All this
149 had to be done in a way that would not affect pure 32-bit builds on
150 platforms that do not support large files. Large file support is
151 summarized in the [52]Table of Platforms; entries in Yellow (32-bit
152 builds that support 64-bit integers) and Green (64-bit builds) support
155 Note that VMS C-Kermit and Kermit 95 for Windows have always been able
156 to transfer large files. However their user interface used 32-bit
157 integers for statistics and the file transfer display. In C-Kermit 9.0
158 Alpha.03, VMS C-Kermit on 64-bit platforms (Alpha and Itanium) should
159 now give correct statistics and progress displays. (We'll see about
162 How to Test Large-File Transfer
164 Several methods are available for testing large-file transfers:
165 * By transferring a real file that is more than 2147483648 bytes long
166 (a file whose length requires more than 31 bits to express); or to
167 be totally sure, that is longer than 4294967296 bytes (32 bits or
168 more). Or to be double super sure, longer than 8589934592 (33
170 * If you don't have such a file or there is not sufficient disk space
171 for such a file, you can create a special kind of file that takes
172 up one block on the disk but appears to be 4.3GB long by compiling
173 and running [53]THIS C PROGRAM on Linux, Solaris, HP-UX, or other
174 Unix platform that supports large files. Kermit or FTP or any other
175 file transfer program will transfer the result (BIGFILE) in such a
176 way as to actually put 4.3GB (or other desired size; see source) on
178 * You can use Kermit's CALIBRATE feature to transfer a large file
179 that doesn't exist. At the receiver, use RECEIVE /CALIBRATE. At the
180 sender, use SEND /CALIBRATE:length, e.g.:
182 (At remote kermit...)
184 C-Kermit> receive /calibrate
185 (Return to local kermit...)
187 C-Kermit> send /calibrate:4300000000
188 This sends a simulated file 4.3GB in length, that does not exist on
189 the sender and will not take up any disk space on the receiver.
190 SEND /CALIBRATE: accepts big numbers only in Kermit versions that
191 support them (this does not include Kermit 95 on Windows). This
192 method tests only Kermit's ability to express and understand large
193 file sizes, but does not test Kermit's file-system interface, since
194 no files are involved.
196 Arithmetic with Large Integers
198 Because large file support requires the availability of a 64-bit signed
199 integer data type, other aspects of C-Kermit were adapted to use it
200 too, most notably Kermit's algebraic expression evaluator and its
201 [54]S-Expression interpreter, on all platforms that support large files
202 (those listed as 64 or 32/64 in the Word column of the [55]table). In
203 fact, every Kermit command that parses a number in any field can now
204 parse a large number on those platforms.
206 S-Expressions can now be forced to operate with integers only, without
207 floating-point conversion or having to explicitly truncate each result;
208 as an example. see the revised [56]Easter date calculation script.
210 FORCE-3 Packet Protocol
212 The Kermit protocol has proven itself over the past 30 years to be
213 robust in terms of surviving harsh transmission environments and
214 delivering the data correctly and completely. In these times of
215 Internet everywhere and error-correcting modems in the few places where
216 the Internet isn't, few people even recall the kinds of difficult
217 conditions that were common when the Kermit protocol was first
218 developed: noisy telephone lines, serial interfaces that drop
219 characters, lack of transparency to control or 8-bit characters,
220 absence of flow control, "bare" modems without error correction.
222 But the Internet is not everywhere, and not all modems are
223 error-correcting. Perhaps the most difficult trial so far for Kermit or
224 any other protocol is the [57]EM-APEX project, in which floats are
225 dropped into the ocean from an aircraft into the path of a hurricane;
226 these floats dive into the water measuring current, temperature, and
227 salinity at different depths and then surface to phone home, sending
228 the data to land stations using Kermit protocol over
229 non-error-correcting 300bps [58]Iridium satellite modems, with high
230 seas and winds battering the floats and heavy ([59]sometimes
231 electrical) storms between the modem and the satellite.
233 Because of the transmission speed and long distances involved, the
234 transfers were very slow. The Kermit software in the floats is
235 [60]Embedded Kermit, which did not implement sliding windows, which
236 would have sped up the flow considerably. John Dunlap, engineer at the
237 University of Washington's Applied Physics Laboratory, undertook the
238 task of adding sliding windows to E-Kermit. For testing, he rigged up a
239 [61]simulator in which Kermit transfers take place over a connection
240 with different amounts of noise and delay. He found that occasionally,
241 a transfer would appear to succeed, but the received file would be
244 According to the Kermit protocol definition, the first packet always
245 has block-check type 1, a 6-bit checksum, which is the only block check
246 type that all Kermit implementations are required to support; thus any
247 Kermit partner can process this packet. This packet itself can
248 negotiate a higher level of checking, such that subsequent packets have
249 (say) block-check type 3, a 16-bit cyclic redundancy check (CRC)
250 encoded as three printable 7-bit ASCII characters. The 16-bit CRC can
251 catch all errors of certain kinds (single-bit, double-bit, bursts of 16
252 bits or less), and more than 99.9984741210937% of all other possible
255 John's simulations revealed that file corruption could occur undetected
256 when the initial packet was corrupted in such a way that a parameter or
257 capability byte was changed and the checksum also changed to make the
258 packet appear to be correct, thus allowing the transfer to proceed with
259 the two Kermit partners out of sync as to packet encoding and
260 interpretation (the chances of two such errors producing a seemingly
261 valid packet are about 1 in 6000 when using the 6-bit checksum). For
262 example, the compression technique might be misnegotiated and then the
263 receiver might store incoming data without decompressing it.
265 The solution is a new option, selected by:
269 to require a type 3 block check (16-bit CRC) on every packet, including
270 the initial ones, thus reducing the probability of a misnegotiation by
271 many orders of magnitude. THIS PARAMETER CAN NOT BE NEGOTIATED. Each
272 Kermit program must be given the "set block 5" command prior to
273 transfer. That's because normally every Kermit program expects the
274 first packet to have a 6-bit checksum, and if the first packet has a
275 3-byte, 16-bit CRC, the packet receiver will think it is corrupted.
277 In practice, however, it is possible to code the packet receiver
278 "cheat" by reading the packet data before verifying the block check.
279 Thus when the receiver is C-Kermit 9.0 or later or E-Kermit 1.7 or
280 later, it is only necessary to give the "set block 5" command to the
281 file sender, and the receiver will check for a FORCE-3 first packet. If
282 the receiver does not support this feature, however, the initial packet
283 will be be rejected (after several retries) and the file transfer will
284 not take place. There is no attempt to "back off" to normal behavior.
286 CAPTION: Table 4. Kermit Protocol Packet Block Check Types
288 Type Command Bytes Status Explanation
289 1 SET BLOCK 1 1 Required in all Kermit implementations. Negotiated.
290 6-bit checksum, suitable for good connections.
291 2 SET BLOCK 2 2 Optional, negotiated. 12-bit checksum. 64 times
292 stronger than type 1.
293 3 SET BLOCK 3 3 Optional, negotiated. 16-bit CRC.
294 BLANK-FREE-2 SET BLOCK 4 2 Optional, negotiated. 12-bit checksum, two
296 FORCE-3 SET BLOCK 5 3 Optional, not negotiated. 16-bit CRC forced all
299 BLANK-FREE-2 is for environments where Kermit packets are treated as
300 lines of text, and in which trailing blanks can be stripped; for
301 example, when transferring files with an IBM mainframe through a 3270
307 Does the strange behavior of Kermit's \%x variables puzzle or annoy
310 Kermit software development has been a collaborative project over the
311 years, with contributions coming in from almost every country and every
312 sector of the economy - academic, corporate, government. Thus not all
313 versions, and not all features of a given version, are a product of
316 One example was the introduction of variables for text substitution,
317 first in a version of MS-DOS Kermit that was sent in by someone
318 somewhere (I could look it up, but no time...) Although the design of
319 the notation for variable names (table below) is mine, the underlying
320 code was contributed. In that code there was only one kind of variable,
321 and if I recall correctly the variable name was a backslash followed by
322 a single letter, for example \a, \b, etc. The contributed code
323 evaluated these variables recursively, meaning if the definition of a
324 variable contained variable references, then these were resolved when
325 dereferencing the variable, and the process would continue as deep down
326 as necessary to resolve the thing fully.
328 This was sometimes handy, but it had one severe drawback: There was no
329 way to use variables in a straightforward way to represent strings that
330 contained literal backslashes; for example, DOS or Windows pathnames.
331 This gave rise to all kinds of quoting rules and conventions (e.g.
332 doubling backslashes or forcing single-level evaluation with
333 \\fcontents()), and also to the introduction of other kinds of
334 variables that were evaluated one level deep, rather than recursively.
336 To accommodate coexistence of different kinds of variables as well as
337 "escape sequences" for representing control and 8-bit characters, the
338 syntax for variable names was extended to include three elements: the
339 leading backslash, then a single character indicating the type of
340 variable, and then the name of the variable in a format corresponding
341 to the type designator, as shown in this somewhat simplified table:
343 CAPTION: Table 1. Variable-name Syntax in Kermit
346 \000 - \255 8-bit character constant (decimal)
347 \d000 - \d255 Alternative notation for 8-bit character (byte) constant
349 \o000 - \o377 8-bit character constant (octal)
350 \x00 - \xff 8-bit character constant (hexadecimal)
351 \%a - \%z Scalar variable, evaluated recursively.
352 \%0 - \%9 Macro argument, scalar, evaluated recursively.
354 \&a[x] Array reference, evaluated recursively (x is any constant or
356 \v(name) Built-in scalar variable, evaluated one level deep.
357 \m(name) User-defined scalar variable, evaluated one level deep.
358 \$(name) An environment variable, evaluated one level deep.
359 \s(name[n:m]) Compact substring notation, evaluated one level deep.
360 \fname(args...) Built-in function with zero or more arguments.
362 \N OUTPUT command only: NUL, ASCII 0
363 \B OUTPUT command only: BREAK (250ms, for serial connections)
364 \L OUTPUT command only: Long BREAK (1.5sec, ditto)
366 Variable names in Kermit are case-independent. The simplifications in
367 the table are that the notation for decimal and octal bytes can have
368 from one to three digits, and can include braces to separate them from
369 text digits, e.g. \7, \{123}, \o{50}. Hex bytes too, except they must
370 always have exactly two hex digits, 0-9a-f. Array indices must be, or
371 must evaluate to, numbers (floating point numbers are truncated).
372 Associative arrays are also available (dynamic arrays with arbitrary
373 text as subscript), but they are really just a variation on \m()
374 variables (read about associative arrays [63]here). Also, there are
375 some alternative notations for compact substring notation.
377 We didn't want to have lots of "distinguished" characters, as the UNIX
378 shell does; one is enough, clarity over brevity. Although the notation
379 can be a bit cumbersome, we can use the \m(name) form to circumvent the
380 overevaluation in most contexts. But macro arguments are always
381 assigned to the \%0-9 variables, and thus always evaluated recursively,
382 making it difficult and confusing to pass (e.g.) Windows pathnames as
383 arguments to macros. The same is true for array elements, especially in
384 contexts where they are used to return results from built-in functions
385 (for example, \fsplit() used to return the elements of a
386 [64]comma-separated value list if any of the values contained
387 backslashes). An even worse scenario is when macro arguments are passed
388 from one macro to another; for some graphic illustrations see
389 [65]Taming the Wild Backslash - Part Deux from the [66]C-Kermit 7.0
392 We can't just change how variables are evaluated because that would
393 break existing scripts. But we can always add Yet Another SET Command:
395 SET COMMAND VARIABLE-EVALUATION { RECURSIVE, SIMPLE }
397 This applies only to \%a-z and \%0-9 variables and to \&a-z[] arrays
398 (since all other kinds of variables are evaluated only one level deep).
399 The default, of course, for backwards compatibility, is RECURSIVE.
400 SIMPLE forces the evaluation of these variables to return their literal
401 contents, without further evaluation:
403 * An exception is made in the case of array subscripts, because
404 changing how they are evaluated could break a lot of scripts, and
405 anyway there should never be any harm in evaluating them
406 recursively because their final value is always (or should be)
407 numeric, not some string that might contain backslashes.
408 * The VARIABLE-EVALUATION setting is on the command stack. Thus you
409 can give this command in a macro, command file, or user-defined
410 function without affecting the calling environment.
411 * The new \frecurse() function forces recursive evaluation of its
412 argument regardless of the VARIABLE-EVALUATION setting. The
413 argument can be any string (or nothing at all); all the variables
414 in the string, even \m() ones, are evaluated recursively:
420 echo \frecurse(\m(xx))
422 echo \frecurse(it's as easy as \m(xx))
423 it's as easy as easy as 1 2 3
425 * The new \v(vareval) built-in variable contains the current setting
426 (recursive or simple) at the current command-stack level.
428 Here's a short script for illustration:
430 define path c:\users\fdc\somefile.txt
431 define test1 { # Normal recursive argument evaluation
434 define test2 { # Simple argument evaluation
442 And here's the result:
444 ?<ERROR:NO_SUCH_FUNCTION:\fdc\somefile.txt()>
445 test2: arg=c:\users\fdc\somefile.txt
447 The first line might seem surprising, but under the normal rules (see
448 table above) \f indicates a function call, with the letters following
449 the 'f' being the name of the function. But there is no function by
450 that name... and if there were, you probably didn't intend to call it!
452 SET COMMAND VARIABLE-EVALUATION SIMPLE has no effect on constants, only
453 on variables. Note how \m(path) is defined. The DEFINE command assigns
454 the literal value of its argument to the named variable (see Table 3
455 below), thus in this case no special syntax is needed. But in other
456 contexts, you must double the backslashes or use the \fliteral()
457 function to use literal backslashes in data:
459 test2 c:\\users\\fdc\\somefile.txt
460 test2 \fliteral(c:\users\fdc\somefile.txt)
462 C-Kermit 9.0 adds a new notation for \fliteral() which also has certain
463 advantages over it: \q(string):
465 test2 \q(c:\users\fdc\somefile.txt)
467 Since \fliteral() is a function, its argument list (the text within
468 parentheses) has special syntax of its own, in which commas and braces
469 are treated specially and introduce another set of quoting problems.
470 \q(string) doesn't have these problems. The only consideration is that
471 parentheses must be balanced or else quoted (preceded by backslash), or
472 represented as numeric character entities (left paren = \40, (right
475 Or else hold the value in a simple variable as we did with \\m(path)
478 SET COMMAND VARIABLE-EVALUATION SIMPLE is a big change and might have
479 repercussions that didn't show up in the initial tests; a lot more
482 On the topic of variables, let's summarize in one place the ways in
483 which values can be explicitly assigned to variables. There is nothing
484 new here except the table itself:
486 CAPTION: Table 2. Variable Assignment in Kermit
488 Command Shorthand Explanation
489 DEFINE name value .name = value The literal value becomes the contents
490 of the named variable; variables names in the value are copied without
491 evaluation. This command is for defining macros that take parameters,
492 as well as for defining simple variables, especially if the values
494 _DEFINE name value Like DEFINE but the name is evaluated before use.
495 ASSIGN name value .name := value The value is evaluated and the result
496 becomes the contents of the named variable.
497 _ASSIGN name value Like ASSIGN but the name is evaluated before use.
498 EVALUATE name expression .name ::= value The expression (in regular
499 algebraic notation) is evaluated arithmetically and the result becomes
500 the contents of the named variable. If the expression contains any
501 variables they are evaluated first.
502 _EVALUATE name expression Like EVALUATE but the name is evaluated
504 INCREMENT name expression Evaluates the variables in the expression,
505 then evaluates the expression arithmetically, and then adds the value
506 to the contents of the named variable, which must be a number or an
507 algebraic expression. If the expression is empty, a value of 1 is used.
508 _INCREMENT name expression Like INCREMENT but the name is evaluated
510 DECREMENT name expression Evaluates the variables in the expression,
511 then evaluates the expression arithmetically, and then subtracts the
512 value from the contents of the named variable, which must be a number
513 or an algebraic expression. If the expression is empty, a value of 1 is
515 _DECREMENT name expression Like DECREMENT but the name is evaluated
517 DECLARE name = list An array declaration can include an initializer
518 list; items in the list are evaluated before assignment. This can be
519 defeated by doubling any backslashes or enclosing individual arguments
521 DO name arguments name arguments When invoking a macro with a DO
522 command (or an implied one), the arguments are evaluated, then assigned
523 to \%1, \%2, etc, and the macro's name to \%0.
524 (SETQ name value) Kermit also includes a mini-[67]LISP interpreter
526 Variables are evaluated automatically in Kermit commands simply by
527 referencing them, according to rules given in Table 1. The following
528 functions can be used to change how a a particular variable is
531 CAPTION: Table 3. Kermit Functions for Evaluating Variables
533 Function Argument Description
534 \fcontents() \%x or \&x[y] Evaluates the variable or array element
535 (which normally would be evaluated recursively) one level deep.
536 \fdefinition() name If the argument is a \%x variable or an array
537 element, it is evaluated to get the name; otherwise the argument is the
538 name. Its definition is returned with no recursion.
539 \m() name Equivalent to \fdefinition().
540 \frecurse() \m(name) Forces recursive evaluation of a macro definition
541 (a.k.a. long variable name). NOTE: \frecurse() can operate on any kind
542 of variable as well as on any string containing any mixture of
545 C-Kermit's RENAME Command
547 C-Kermit's RENAME command, which is used for changing the names of
548 local files or for moving files locally, has two basic forms:
550 RENAME [ optional-switches ] oldfilename newfilename
551 This form lets you change the name of a single file from
552 oldfilename to newfilename. Example:
553 rename thismonth.log lastmonth.log
555 RENAME [ optional-switches ] filespec directoryname
556 This form lets you move (without renaming) one or more files
557 (all the files that match the filespec, which may contain
558 wildcard characters such as "*") to the given directory.
560 rename *.txt ~/textfiles/
562 Traditionally, the optional switches have been:
564 RENAME /LIST oldname newname
565 Display the old and new name for each file while renaming.
566 Synonyms: /LOG, /VERBOSE. Example:
567 rename /list *.txt ~/textfiles/
569 RENAME /NOLIST oldname newname
570 Don't display the old and new name for each file while renaming.
571 This is the default behavior. Synonyms: /NOLOG, /QUIET. Example:
572 rename /nolist *.txt ~/textfiles/
574 Reminder: Every switch starts with a slash (/) and must be preceded by
577 New RENAME Features for C-Kermit 9.0
579 A series of new options (switches) have been added to let you change
580 the names of multiple files at once by case conversion, string
581 substitution, or character-set conversion, and optionally also move
582 them to a different directory:
584 /LOWER: Convert the filename to lowercase
585 /UPPER: Convert the filename to uppercase
586 /CONVERT: Change the filename's character encoding
587 /REPLACE: Do string substitutions on the filename
589 If the source-file specification includes a path or directory, any
590 changes are applied to the filenames only, not to the directory or path
593 Since name changes, when applied to many files at once, can have
594 consequences that are not easily undone, there are also some new
595 controls, safeguards, and conveniences:
598 This switch tells Kermit to show you what the RENAME command
599 would do without actually doing it. /SIMULATE implies /LIST.
601 RENAME /COLLISION:{FAIL,SKIP,OVERWRITE}
602 This switch governs Kermit's behavior when renaming multiple
603 files, and any of the names would collide with the name of a
604 file that already exists. The default, for compatibility with
605 earlier releases of C-Kermit, is OVERWRITE, i.e. write over the
606 existing file. The other two protect existing files. SKIP means
607 to skip (not rename) the file that would cause the collision,
608 and proceed to the next file, if any. FAIL means that no files
609 will be renamed if there would be any collisions; for this
610 Kermit makes two passes, checking each new name it constructs
611 for existence before starting the second pass (however, there is
612 no guarantee that in the second pass, it won't create the same
613 new name for more than one file; in that case, it will stop
614 before executing the second rename). Example:
615 rename /simulate /collision:proceed * ~/tmp/
617 Reminder: In switches such as /COLLISION that take arguments
618 (operands), the switch name and its argument(s) are separated by a
619 colon (:) with no intervening spaces. Also remember that Kermit
620 keywords can always be abbreviated by leaving off characters from the
621 right, as long as the result is still unique in its context. Thus "ren
622 /col:f" would be equivalent to "rename /collision:fail".
624 You can change the following preferences for the RENAME command with
625 the new SET RENAME command:
627 SET RENAME LIST { ON, OFF }
628 Tells the RENAME command whether to list its actions if you
629 don't include a /LIST or /NOLIST or equivalent switch.
631 SET RENAME COLLISION { FAIL, OVERWRITE, SKIP }
632 Tells the RENAME command how to handle filename collisions in
633 the absence of a /COLLISION switch. That is, it replaces the
634 default action of OVERWRITE with action of your choosing, which
635 is then used in any RENAME command that does not include an
636 explicit /COLLISION switch.
639 Displays the current SET RENAME settings.
641 Changing the Case of Filenames
643 RENAME /UPPER:{ALL,LOWER} filespec [ directory ]
644 RENAME /LOWER:{ALL,UPPER} filespec [ directory ]
645 These switches let you change the alphabetic case of letters in
646 all the files whose names match the filespec. If a directory
647 name is given after the filespec, then the files are also moved
648 to the given directory.
650 By default, all files that match the given filespec have their names
651 changed (if necessary). This is what the ALL argument means, e.g.:
656 You can use either form: RENAME /LOWER is equivalent to RENAME
657 /LOWER:ALL. The other argument (/LOWER:UPPER or /UPPER:LOWER) means to
658 leave mixed-case filenames alone, and rename only those files whose
659 names contain letters of only the given case. Examples:
661 RENAME /UPPER:ALL foo.bar
662 Changes the filename to FOO.BAR.
664 RENAME /UPPER foo.bar
665 Same as "rename /upper:all foo.bar".
667 RENAME /UPPER foo.bar ~/old/
668 Renames foo.bar to FOO.BAR and moves it to the user's old
672 Changes the names of all files to have only lowercase letters.
674 RENAME /LOWER:UPPER *
675 Changes the names of only those files whose names contain no
676 lowercase letters to have only lowercase letters. For example,
677 FOO.BAR would be changed, Foo.Bar would not be changed. foo.bar
678 would not be changed either because it's already all lowercase.
680 RENAME /LOWER:UPPER * ~/new/
681 Same as the previous example, but also moves each file to the
682 user's new directory (whether it was renamed or not).
684 Case conversion works reliably for ASCII characters only. Kermit uses
685 the C library for this, which on any given platform might or might not
686 handle non-ASCII letters, and if it does, then how it works would
687 normally depend on your locale definitions (the LC_CTYPE and/or LANG
688 environment variable in Unix). When non-ASCII letters are not handled
689 by the C library, the RENAME command does change their case. For
690 example, Olga_Tañón.txt might become OLGA_TAñóN.TXT.
692 String Replacement in Filenames
694 The RENAME command also lets you change filenames by string
697 RENAME /FIXSPACES[:String] filespec [ directory ]
698 Replaces all spaces in each matching filename by the given
699 string, if any, or if none is given, by underscore. Examples:
702 RENAME /FIXSPACES:_ *
703 RENAME /FIXSPACES:"" *
704 RENAME /FIXSPACES:<040> *
706 The first two are equivalent, replacing each space with
707 underscore; a file called "My Favorite Photo.jpg" becomes
708 "My_Favorite_Photo.jpg". The third example removes all spaces
709 ("MyFavoritePhoto.jpg"). The fourth replaces each space with the
710 string "<040>" ("My<040>Favorite<040>Photo.jpg").
712 RENAME /REPLACE:{{String1}{String2}} filespec [ directory ]
713 Renames each matching file by changing occurrences of String1 in
714 its name to String2. If a directory specification is included,
715 the file is also moved to the given directory (even if the name
716 was not changed). Note that in this case, the curly braces are
717 part of the command. Example:
719 RENAME /REPLACE:{{.jpeg}{.jpg}} *
721 changes all *.jpeg files to *.jpg.
723 By default, RENAME /REPLACE changes all occurrences of String1 in each
724 filename to String2 so, for example, if you had a file called
725 abcjpegxyz.jpeg, the command just shown would change its name to
728 For greater control and flexibility, the /REPLACE: switch argument can
729 take several distinct forms:
731 RENAME /REPLACE:String1 filespec [ directory ]
732 This means to remove all occurrences of String1 from the given
733 filenames name. It is equivalent to /REPLACE:{{String1}{}}. A
734 handy use for this option is to remove spaces from filenames.
736 RENAME /REPLACE:{{String1}{String2}} filespec [ directory ]
737 As already noted, this replaces every occurrence of String1 with
738 String2 in each filename. Alphabetic case in string matching is
739 done according to the current SET CASE setting.
741 RENAME /REPLACE:{{ }{_}} filespec [ directory ]
742 This replaces all spaces in the given filenames with underscore,
743 equivalent to RENAME /FIXSPACES.
745 RENAME /REPLACE:{{String1}{String2}{Options}} filespec [ directory ]
746 Options can be included that add more control to the process.
747 The option string is a sequence of characters; each character in
748 the string is an option. The choices are:
750 A String matching is to be case-sensitive, regardless of SET CASE.
751 a String matching is to be case-independent, regardless of SET CASE.
752 ^ String replacement will occur only at the beginning of the filename.
753 $ String replacement will occur only at the end of the filename.
754 1 Only the first occurrence of the string will be replaced.
755 2 Only the second occurrence of the string will be replaced.
757 9 Only the ninth occurrence of the string will be replaced.
758 - (hyphen, minus sign) Before a digit: occurrences will be counted from
760 ~ (tilde) Before digit or minus sign: all occurrences but the given one
763 The tilde modifier works only with single-byte character sets such as
764 ASCII, CP437, ISO 8859-1, etc, but not with multibyte character sets
765 such as UCS2, UTF8, or any of the Japanese Kanji sets.
767 Here are some examples showing how to use the /REPLACE options:
769 RENAME /REPLACE:{{foo}{bar}{^}} *
770 For all files whose names start with "foo", replaces the "foo"
771 at the beginning with "bar".
773 RENAME /REPLACE:{{}{New-}{^}} *
774 Prepends "New-" to the name of each file.
776 RENAME /REPLACE:{{.jpeg}{.jpg}{$}} *
777 Replaces ".jpeg" at the end of each filename with ".jpg".
779 RENAME /REPLACE:{{}{-Old}{$}} *
780 Appends "-Old" to the name of each file.
782 RENAME /REPLACE:{{foo}{bar}{a}} *
783 Replaces "foo", "FOO", "Foo", "fOO", etc, with "bar" in each
786 RENAME /REPLACE:{{foo}{bar}{A}} *
787 Replaces only (lowercase) "foo" in filenames with "bar".
789 RENAME /REPLACE:{{a}{XX}} *
790 Changes every "a" to "XX". For example a file called "a.a.a.a"
791 would become "XX.XX.XX.XX".
793 RENAME /REPLACE:{{a}{X}{2}}
794 Changes only the second "a" to "X". For example a file called
795 "a.a.a.a" would become "a.X.a.a".
797 RENAME /REPLACE:{{a}{X}{-1}}
798 Changes only the final "a" in the filename (it doesn't have to
799 be at the end) to "X". For example a file called "a.b.a.c.a.d"
800 would become "a.b.a.c.X.d".
802 RENAME /REPLACE:{{foo}{NOTFOO}{-2}}
803 Changes the second-to-last "foo" (if any) in the filename to
806 RENAME /REPLACE:{{foo}{}{-2}}
807 Deletes the second-to-last "foo" (if any) from the filename.
809 RENAME /REPLACE:{{.}{_}{~1}}
810 Changes all but the first period to an underscore; for example,
811 "a.b.c.d.e" would become "a.b_c_d_e".
813 RENAME /REPLACE:{{.}{_}{~-1}}
814 Changes all but the final period to an underscore; for example,
815 "a.b.c.d.e" would become "a_b_c_d.e".
817 In the Options field, digits (and their modifiers), ^, and $ are
818 mutually exclusive. If you include more than one of these in the option
819 string, only the last one is used. Similarly for 'a' and 'A':
821 RENAME /REPLACE:{{foo}{bar}{Aa2$^}} *
822 This replaces "foo" with "bar" no matter what combination of
823 upper and lower case letters are used in "foo" ('a' overrides
824 'A' in the option string), but only if "foo" is at the beginning
825 of the filename ('^' overrides '$' and '2').
827 If you give an /UPPER or /LOWER switch and a /REPLACE switch in the
828 same RENAME command, the /REPLACE action occurs first, then the case
831 RENAME /REPLACE:{{foo}{bar}} /UPPER * /tmp
832 For each file: changes all occurrences of "foo" in the name to
833 "bar", then converts the result to uppercase, and then moves the
834 file to the /tmp directory. So (for example) "foot.txt" would
835 become "/tmp/BART.TXT".
837 Changing the Character Encoding of Filenames
839 As you know, text is represented on the computer as a series of
840 numbers, with a given number corresponding to a given character
841 according to some convention or standard. Filenames are represented the
842 same way. The trouble is, different computers, or even different
843 applications on the same computer, might use different standards or
844 conventions ("character sets") for representing the same characters.
845 Usually ASCII is safe, but anything beyond that -- non-ASCII characters
846 such as accented or non-Roman letters -- is likely to vary. Sometimes
847 you have text that's in the "wrong" character set and you need to
848 convert it to something you can can use. Kermit has always been able to
849 handle this as part of file transfer and terminal emulation, as well as
850 being able to convert text files locally with its TRANSLATE command.
851 Now there's a way to convert filenames too, for example after copying
852 files from a CD that uses a different encoding:
854 RENAME /CONVERT:charset1:charset2 filespec [ directory ]
855 Converts filenames from the first character set to the second
856 one. The two character sets can be chosen from the SET FILE
857 CHARACTER-SET list; for complete details see [68]this page. For
858 example suppose you have a file called "Olga_Tañón.txt" on a
859 computer where ISO 8859-1 Latin Alphabet 1 is used, and you have
860 transported it (e.g. on CDROM) to another computer where the
861 text encoding is UTF8. Maybe you also have a lot of other files
862 with similar names in the same directory. You can convert the
863 filenames to UTF8 like this:
865 RENAME /CONVERT:latin1:utf8 *
867 /CONVERT can not be combined with /UPPER, /LOWER, or /REPLACE.
869 You should NOT use UCS2 for filenames since this encoding is not
870 compatible with C strings used in Unix and elsewhere.
872 RENAME /CONVERT affects only the filename, not the file's contents. You
873 can use the TRANSLATE command to convert the encoding of the contents
878 See the [69]C-Kermit Daily Builds page for details. Very briefly:
880 * Perhaps most important, modernized makefile targets for the major
881 Unix platforms: Linux, Mac OS X, AIX, Solaris, etc. These are
882 somewhat automated; not autoconf exactly, but they cut down
883 significantly on redundant targets. For example, one single "linux"
884 target works on many (hopefully all) different Linux
885 configurations, where before different targets were required for
886 different combinations of (e.g.) curses / ncurses / no curses;
887 32-bit / 64-bit; different feature sets and library locations.
888 (Separate targets are still required for Kerberos and/or SSL
889 builds, but they are "subroutinized".)
890 * Bigger buffers, more storage for commands, macros, scripts,
891 strings, and filename expansion in 64-bit versions and in 32-bit
892 versions that support large files.
893 * User-settable FTP timeout, works on both the data and control
895 * FTP access to ports higher than 16383.
896 * Built-in FTP client for VMS. This is the [70]same FTP client Unix
897 C-Kermit has had since version 8.0, minimally adapted to VMS by
898 SMS, supporting binary and Stream_LF file transfer only (in other
899 words, nothing to handle RMS files), but otherwise fully functional
900 (and scriptable) and theoretically capable of making connections
901 secured by SSL (at least it compiles and links OK with SSL - HP SSL
903 * Large file support in VMS, also by SMS. Alpha and Itanium only (not
904 VAX). VMS C-Kermit was already able to transfer large files, but
905 the file-transfer display (numbers and progress bar) and statistics
906 were wrong because they used ints. In the present Alpha test
907 release, this is an optional feature requested by including the "f"
909 * New PUTENV command that allows Kermit to pass environment variables
910 to subprocesses (Unix only, "help putenv").
911 * New TOUCH command, many file selection options ("help touch").
912 * New DIRECTORY command options and switches (/TOP, /COUNT;
913 HDIRECTORY, WDIRECTORY...). To see the ten biggest files in the
914 current directory: "dir /top:10 /sort:size /reverse *" or
915 equivalently, "hdir /top:10 *". WDIR lists files in reverse
916 chronological order, shorthand for "dir /sort:date /reverse".
917 * New command FSEEK /FIND:string-or-pattern, seeks to the first line
918 in an FOPEN'd file that contains the given string or matches the
919 given pattern. Example: Suppose you have a file of lines like this:
921 quantity description...
922 in which the first "word" is a number, followed by a description
923 (for example, the name of an item). Here is how to use FSEEK to
924 quickly get the total quantity of any given item, which is passed
925 as a parameter (either a literal string or a pattern) on the
928 #!/usr/local/bin/kermit +
929 if not def \%1 exit 1 Usage: \fbasename(\%0) string-or-pattern
931 .filename = /usr/local/data/items.log # Substitute the actual filename
932 set case off # Searches are case-independent
933 fopen /read \%c \m(filename) # Open the file
934 if fail exit 1 "\m(filename): \v(errstring)" # Fail: exit with error message
935 .total = 0 # OK: Initialize the total
936 echo Searching "\%1"...
939 fseek /line /relative /find:\%1 \%c 0 # Get next line that has target
940 if fail break # Failure indicates EOF
941 fread /line \%c line # Read it
942 if fail break # (shouldn't happen)
943 increment total \fword(\m(line),1) # Increment the total
945 fclose \%c # Close the file
946 echo Total for "\%1" : \m(total) # Print the result
949 The syntax of the FSEEK command in this example indicates that each
950 search should start relative to the current file line. Since Kermit
951 is an interpretive language, FSEEK is a lot faster than FREAD'ing
952 each line and checking it for the target, especially for big files.
953 An especially handy use for FSEEK is for use with potentially huge
954 sequentially timestamped logs, to seek directly to the date-time
955 where you want to start processing. Some other improvements for the
956 FOPEN/FREAD/FWRITE/FCLOSE family of commands are included also
957 (performance, bug fixes, convenience features), listed in the
958 [71]change log. (Prior to 9.0.299 Alpha.02, the FSEEK /FIND:
959 command always started from the top.)
960 * MIME synonyms for character-set names: A new equivalence between
961 MIME names and Kermit names for character sets, with a new table
962 showing the supported sets [72]HERE (this feature is also
963 illustrated in the [73]Weblog script).
964 * Unix C-Kermit SET TERMINAL TYPE now passes its arguments to
965 subprocesses as an environment variable.
966 * SET SESSION-LOG TEXT now strips out ANSI escape sequences from the
968 * For interacting with POP servers over clear-text or SSL-secured
970 + New SSL and TLS "raw" connections (no Telnet protocol).
971 + New INPUT command options for reading and capturing (perhaps
972 while scanning) continuous incoming text, such as INPUT
973 /NOWRAP (explained [74]HERE).
974 + New \femailaddress() command to extract the e-mail address
975 from an Internet mail message To: or From: line, used in
976 fetching mail from POP servers.
977 + Improved date parsing commands and functions for parsing the
978 different date formats that can appear in e-mail.
979 + Production scripts for fetching mail from a secure POP server,
981 * Various features added to make Kermit more useful for writing CGI
982 scripts such as INPUT /COUNT:n to INPUT exactly n characters
983 (useful for reading form data).
984 * New \fpictureinfo() function for getting orientation and dimensions
985 of JPG and GIF images, described [76]HERE.
986 * New \fgetpidinfo() function for testing whether a given process
988 * \fkwdvalue() function fixed to allow multiword values.
989 * New function \fcount(s1,s2) to tell the number of occurrences of s1
991 * New \flopx() function returns rightmost field from string (such as
993 * New function \ffunction(s1) to tell whether a built-in s1 function
995 * New \fsqueeze(s1) function removes leading and trailing whitespace
996 from string s1, changes tabs to spaces, squeezing each run of
997 repeated whitespace characters to a single space.
998 * Compact substring notation: \s(somestring[12:18]) is the same as
999 \fsubstring(\m(somestring),12,18), i.e. the substring starting at
1000 position 12, 18 characters long. \s(somestring[12_18]) means
1001 characters 12 through 18 of the string (7 characters). Also,
1002 \s(somestring[17.]) returns character number 17 of somestring.
1003 * The string indexing functions now accept an optional trailing
1004 argument specifying the occurrence number of the target string.
1005 Likewise, \fword() can fetch words from the right as well as the
1007 * The COPY command in Unix C-Kermit has a new /PRESERVE switch,
1008 equivalent to Unix "cp -p".
1009 * ASKQ /ECHO:c can be used to make the characters the user types echo
1010 as the character c, e.g. asterisk when typing a password.
1011 * IF LINK filename to test if the filename is a symlink.
1012 * Ctrl-K, when typed at the command parser, replaces itself with most
1013 recently entered file specification.
1014 * In Unix, the ability to log a terminal session to a serial port,
1015 for use with speaking devices or serial printers; described
1016 [77]HERE. Also for the same purpose, SET SESSION-LOG
1017 NULL-PADDED-LINES for a speech synthesizer than needed this.
1018 * Adaptation to OpenSSL 0.9.8 and 1.0.0.
1019 * Lifted the restriction on having a remote Kermit program send
1020 REMOTE commands to the local. A very big ex-client needed to be
1021 able to do this (branches would connect to headquarters and upload
1022 files; HQ would then download patches, a REMOTE HOST command was
1023 necessary to allow the remote headquarters machines to install the
1024 patches on the local client; of course the client first has to
1025 ENABLE HOST because this is a risky scenario). The reason for the
1026 restriction was that the server, upon receiving any REMOTE command
1027 would send the results (output) back to the client as a file
1028 transfer with "destination screen", but of course the remote has no
1030 * Added XMESSAGE, which is to [78]MESSAGE as XECHO is ECHO: it
1031 outputs a string with no line terminator DEBUG MESSAGE is ON.
1032 * Fixed \frecurse() to not dump core when invoked with no arguments.
1033 * Improved text for HELP FUNCTION SPLIT and HELP FUNCTION WORD.
1034 * Patches for Debian 6.0 "Squeeze" from Ian Beckwith.
1035 * \fcontents(\&a[3]) got an error if the array was declared but its
1036 dimension was less than 3. Now it simply returns and empty string.
1037 * \fsplit(), when parsing lines from CSV and TSV files, was treating
1038 backslash in the data the same way it treats backslash in Kermit
1039 commands. This was fixed to treat backslash like any other
1041 * Builds for Solaris 9 and later now use streams ptys rather then the
1042 old BSD-style ptys. Thanks to Gary Mills for this one, who noticed
1043 that he couldn't have more than 48 C-Kermit SSH sessions going at
1044 once and figured out why.
1045 * As noted [79]below DES encryption is being retired from many
1046 platforms and libraries that once used it. I changed the Solaris
1047 and Linux OpenSSL builds to account for this by testing for it. I
1048 probably should also add a OMITDES option to omit DES even if it is
1049 installed, but "KFLAGS=-UCK_DES" seems to do the job for now.
1050 * I changed the Linux build to test for the OpenSSL version (like the
1051 Solaris version already did), rather than assuming OpenSSL 0.9.7.
1052 * A couple minor changes for Tru64 Unix 5.1B from Steven Schweda but
1053 we still have some trouble on that platform. As a workaround "make
1054 osf1" can be used there.
1055 * Unix makefile and man page are now included in the Zip
1057 * \fjoin(), which is the inverse function of fsplit() now accepts CSV
1058 and TSV as a second argument, to transform an array into a
1059 comma-separated or tab-separated value list, as described [80]HERE.
1060 * Even in 2010, Unix distributions continue to change their UUCP
1061 lockfile conventions. C-Kermit 9.0 contains support from Joop
1062 Boonen for OpenSuSE >= 11.3 and recent Debian, which no longer have
1063 baudboy.h, which first appeared in Red Hat 7.2 in 2003.
1064 * From Lewis McCarthy:
1066 Based on code inspection, C-Kermit appears to have an SSL-related
1067 security vulnerability analogous to that identified as CVE-2009-3767
1069 [81]http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2009-3767).
1071 I'm attaching a patch for this issue relative to the revision of
1072 ck_ssl.c obtained from a copy of
1073 [82]http://www.columbia.edu/kermit/ftp/test/tar/x.zip downloaded on
1074 2010/07/30, which I believe is the latest.
1075 When this flaw was first widely publicized at last year's Black Hat
1076 conference, it was claimed that some public certificate authorities
1077 had indeed issued certificates that could be used to exploit this
1078 class of vulnerability. As far as I know they have not revealed
1079 specifically which public CA(s) had been found issuing such
1080 certificates. Some references:
1081 + [83]http://www.mseclab.com/?p=180
1082 + [84]http://www.theregister.co.uk/2009/07/30/universal_ssl_cert
1085 * Peter Eichhorn reported that "RENAME ../x ." didn't work; fixed
1087 * If only one file is FOPEN'd, FCLOSE given with no arguments would
1088 close it; this was a "convenience feature" that turned out to be
1089 dangerous. For safety FCLOSE has to require a specific channel
1090 number or the word ALL.
1091 * Added \fstrcmp(s1,s2,case,start,length), which has the advantage
1092 over IF EQU,LGT,LLT that case sensitivity can be specified as a
1093 function arg, and also substrings can be specified.
1094 * New built-in functions:
1096 \fcvtcsets(string,cs1,cs2)
1097 Function to convert a string from one character set to
1100 \fdecodehex(string[,prefix])
1101 Function to decode a string containing hex escapes.
1103 \fstringtype(string)
1104 Function to tell whether a string is 7-bit, 8-bit, or
1107 For the motivation for these features and an application that uses
1108 them to analyze web logs, see the Weblog script below.
1111 Lazy IF Conditions: Now you can do this:
1112 define foo some number
1116 define foo some number
1119 Of course the old way still works too. But watch out because if the
1120 variable name is the same as a symbolic IF condition (for example
1121 COUNT), it won't do what you expected. (IF COUNT was used for loop
1122 control in early versions of MS-DOS Kermit, before it got real FOR
1123 and WHILE loops; it was added to C-Kermit for compatibility, and it
1124 can't be removed because that could break existing scripts).
1125 * Escape sequences are now stripped from text-mode session logs not
1126 only in CONNECT sessions but also in whatever is logged by the
1127 INPUT command; described in the [85]next section.
1128 * New commands for selectively issuing progress or debugging messages
1129 from scripts, also described in the next section.
1130 * Fix from [86]John Dunlap to prevent the fixed packet-timeout
1131 interval from going to an unexpected value.
1132 * Alpha.04 fixes a problem with FTP connections made from 64-bit Unix
1133 platforms. All the other changes in this section were to Alpha.03.
1134 * Relaunching a closed SSH connection with the CONNECT command is now
1135 possible, as it always has been with Telnet and other connection
1136 types; suggested by Peter Eichhorn (needs testing).
1137 * A symbol conflict fixed that prevented successful build on
1139 * Fixes from Christian Corti for building on SunOS 4.1.
1140 * New aixg target for building on AIX with gcc.
1141 * New aix+ibmssl target. This is nice because the IBM-supplied SSL
1142 libraries and header files are in a known location; no need to
1143 [88]set environment variables giving their locations.
1144 * "Large File Support" is now included by default on Alpha and IA64
1145 hardware on VMS 7.3 and later, and it should work much better than
1147 * Kermit's internal FTP client is now included by default in any
1148 build that also includes TCP/IP networking. At present, the FTP
1149 client seems to work well for binary-mode transfers; text (ASCII)
1150 mode transfers still need some work. In builds that also include
1151 Secure Sockets Layer (SSL) security (next item) the FTP client
1152 should be able to make securely authenticated and encrypted
1154 * In network builds that request OpenSSL support, e.g.:
1156 $ @ckvker "" "" "CK_SSL"
1157 the OpenSSL version is detected automatically and the appropriate
1158 compile-time options are emitted (such as
1159 OPENSSL_DISABLE_OLD_DES_SUPPORT).
1160 * Preliminary / limited support for the ODS-5 file system on VMS 7.2
1161 and later, Alpha and Itanium only (needs testing): Filenames can be
1162 mixed case and can be longer.
1163 * Support for older and older VMS versions.
1164 * In the VMS build procedure, CKVKER.COM, the "i" option in P1 now
1165 means don't include the internal FTP client, and the "f" option
1166 means do not include "Large File" support. Large File support in
1167 VMS really only applies to the file-transfer display and
1168 statistics, which would go out of whack as soon as the byte count
1169 overflowed 31 bits because this is C-Kermit, built with the C
1170 compiler and the C library (runtime system), which did not support
1171 long integers until VMS 7.3.
1172 * The [89]LISP Operator ROUND now takes an optional second argument
1173 that specifies the number of places to round to, e.g.
1174 (ROUND dollars 2) rounds dollars to 2 decimal places.
1175 * Improved pattern matching in many commands for both strings and
1177 * Various minor new features, plus numerous bug fixes and speedups.
1181 A top priority for new Kermit software releases has always been
1182 backwards compatibility. A script written for a previous Kermit release
1183 should run the same way in the new release.
1185 There's one exception this time. The [90]\fsplit() function is
1186 incredibly handy, it can do almost anything, up to and including
1187 parsing a LISP program (the underlying code is the basis of the
1188 [91]S-Expression interpreter). But did you ever try to use it to parse
1189 (say) a Tab-Separated-List (TSV file) or Comma-Separated-List (CSV)? It
1190 works as expected as long as the data contains only 7-bit characters.
1191 But if your data contains (say) Spanish or German or Russian text
1192 written in an 8-bit character set such as ISO 8859-1, every 8-bit
1193 character (any value 128-255) is treated as a break character. This is
1194 fixed in C-Kermit 9.0 by treating all 8-bit bytes as "include"
1195 characters rather than break characters, a total reversal of past
1196 behavior. I don't think it will affect anyone though, because if this
1197 had happened to anyone, I would have heard about it!
1199 Since most standard 8-bit character sets have control characters in
1200 positions 128-160, it might have made sense to keep 128-160 in the
1201 break set, but with the proliferation of Microsoft Windows code pages,
1202 there is no telling which 8-bit character is likely to be some kind of
1203 text, e.g. "smart quotes" or East European or Turkish accented letters.
1205 What's Not In C-Kermit 9.0
1207 Some large projects that were contemplated have not been done,
1209 * IPv6. Honestly, there has been zero demand for this, and it would
1210 be a lot of work and disruption to the code base. Volunteers
1211 welcome, I guess. It could be a CS project.
1212 * A database interface - MySQL or ODBC. For this one, there is some
1213 demand but I haven't had a chance to even look into it.
1214 * There's a looming issue with DES encryption; major vendors are
1215 removing it from their platforms, starting with Apple in Mac OS X
1216 10.6, with Microsoft to follow suit. A secure version of Kermit can
1217 be built without DES, but in limited testing successful connections
1218 were spotty (e.g. with Kerberos 5).
1219 * Cleaning up the Unix makefile. It has 25 years' worth of targets in
1220 it. It is very likely safe to remove most of them, since (a) most
1221 old platforms have gone away by now, or have been upgraded, due to
1222 hacking vulnerabilities; (b) the market has consolidated
1223 considerably; and (c) most of the new features of C-Kermit 9.0,
1224 such as large files, won't be of any use on older platforms and
1225 previous C-Kermit versions will remain available.
1226 * Packages. Everybody wants an install package custom made for their
1227 own computer, Linux RPMs being the prime example but far from the
1228 only one. These will come, I suppose (especially with some Linux
1229 sites having a policy against installing any application that does
1230 not come as an RPM). In the meantime, here's a page that describes
1231 some Kermit-specific issues in package construction:
1232 [92]ckpackages.html.
1235 Using External File-Transfer Protocols on Secure Connections
1237 After C-Kermit 8.0.212 Dev.27 (2006/12/22), I spent a big chunk of time
1238 trying to solve a particular problem that some of you have complained
1239 about and others might be familiar with: If you use C-Kermit to make a
1240 secure Telnet connection to another host (e.g. with Telnet SSL/TLS,
1241 Kerberos, or SRP) and then attempt to transfer a file using an external
1242 protocol such as Zmodem, it doesn't work.
1244 That's because as coded (through 8.0.211), C-Kermit simply starts the
1245 external protocol in a fork with its standard i/o redirected to the
1246 connection. This completely bypasses the encryption and decryption that
1247 is done by C-Kermit itself, and of course it doesn't work. The same
1248 thing occurs if you use the REDIRECT command. The routine that handles
1249 this is ttruncmd() in ckutio.c.
1251 In order to allow (say) Zmodem transfers on secure connections, it is
1252 necessary for C-Kermit to interpose itself between the external Zmodem
1253 program and the connection, decrypting the incoming stream before
1254 feeding it to Zmodem and encrypting Zmodem's output before sending out
1257 In principal, this is simple enough. We open a pseudoterminal pair
1258 ("master" and "slave") for Zmodem's i/o and we create a fork and start
1259 Zmodem in it; we read from the fork pty's standard output, encrypt, and
1260 send to the net; we read from the net, decrypt, and write to the fork
1261 pty's standard input.
1263 In practice, it's not so simple. First of all, pseudoterminals (ptys)
1264 don't seem to interface correctly with certain crucial APIs, at least
1265 not in the OS's I have tried (Mac OS X, Linux, NetBSD, etc), such as
1266 select(). And i/o with the pty often - perhaps always - fails to
1267 indicate errors when they occur; for example, when the fork has exited.
1269 But, even after coding around the apparent uselessness of select() for
1270 multiplexing pty and net, and using various tricks to detect when the
1271 external protocol exits and what its exit status is, I'm still left
1272 with a show-stopping problem: I just simply can not download (receive)
1273 a file with Zmodem, which is the main thing that people would probably
1274 want to do. I can send files just fine, but not receive. The incoming
1275 stream is delivered to Zmodem (to the pty slave) but upon arrival at
1276 the Zmodem process itself, pieces are always missing and/or corrupt.
1277 Yet I can receive files just fine if I use Kermit itself (C-Kermit or
1278 G-Kermit) as the external protocol, rather than Zmodem.
1280 I can think of two reasons why this might be the case:
1282 1. Zmodem sends all 8-bit bytes and control codes in the clear, and
1283 maybe the pty is choking on them because it thinks it is a real
1286 But Zmodem puts its controlling terminal into raw mode. And C-Kermit
1287 puts the pty into raw mode too, just for good measure. If any 0xFF
1288 codes are in the Zmodem data stream, and it's a Telnet session, Kermit
1289 does any needed byte stuffing/unstuffing automatically. Anyway, if I
1290 tell Zmodem to prefix everything, it makes no difference.
1292 2. Zmodem is a streaming protocol and perhaps the pty driver can't
1293 keep up with a sustained stream of input at network speeds. What
1294 would be the method of flow control?
1296 I can vary the size of the i/o buffers used for writing to the pty, and
1297 get different effects, but I am not able to get a clean download, no
1298 matter what buffer size I use. write()'ing to the pty does not return
1299 an error, and I can't see the errors because they happen on the master
1300 side. It's as if the path between the pty slave and master lacks flow
1301 control; I deliver a valid data stream to the pty slave and the master
1302 gets bits and pieces. This impression is bolstered somewhat by the
1303 "[93]man 7 pty" page in HP-UX, which talks about some special modes for
1304 ptys that turn off all termio processing and guarantee a
1305 flow-controlled reliable stream of bytes in both directions - a feature
1306 that seems to be specific to HP-UX, and exactly the one we need
1309 Well, in Pass One I used C-Kermit's existing pty routines from
1310 ckupty.[ch], which are well-proven in terms of portability and of
1311 actually working. They are currently used by SET HOST /PTY for making
1312 terminal connections to external processes. But these routines are
1313 written on the assumption that the pty is to be accessed interactively,
1314 and maybe they are setting the fork/pty arrangement up in such a way
1315 that that's not suitable for file transfer. The Pass One routine is
1316 called xttptycmd() in ckutio.c.
1318 So in Pass Two I made a second copy of the routine, yttptycmd(), that
1319 manages the pty and fork itself, so all the code is in one place and
1320 it's simple and understandable. But it still doesn't work for Zmodem
1321 downloads. In this routine, I use openpty() to get the pty pair, which
1322 is not portable, so I can have access to both the master and slave pty
1323 file descriptors. This version can be used only a platforms that have
1324 openpty(): Linux, Mac OS X, NetBSD, etc.
1326 In Pass Three, zttptycmd(), I tried using pipes instead of ptys, in
1327 case ptys are simply not up to this task (but that can't be true
1328 because if I make a Telnet or SSH connection into a host, I can send
1329 files to it with Zmodem, and the remote Zmodem receiver is, indeed,
1330 running on a pty). But pipes didn't work either.
1332 In Pass Four, I extracted the relevant routines into a standalone
1333 program based on yttptycmd() (the openpty() version, for simplicity),
1334 which I tested on Mac OS X, the idea being to rule out any
1335 "environmental" effects of running inside the C-Kermit process. There
1336 was no difference -- Kermit transfers (with C-Kermit itself as the
1337 external protocol) worked; Zmodem transfers (neither sz or lsz) did
1340 Well, it's a much longer story. As the external protocol, I've tried
1341 rzsz, crzsz, and lrzsz. We know that some of these have quirks
1342 regarding standard i/o, etc, which is one of the reasons for using ptys
1343 in the first place, and i/o does work - just not reliably. Anyway, the
1344 1100 lines or so of [94]ckc299.txt, starting just below where it says
1345 "--- Dev.27 ---" tell the full story. At this point I have to give up
1346 and move on; it might be more productive to let somebody else who has
1347 more experience with ptys take a look at it - if indeed anyone still
1348 cares about being able to do Zmodem transfers over secure Telnet
1351 C-Kermit 9.0 contains the three new routines (and some auxiliary ones),
1352 but they are not compiled or called unless you build it specially:
1354 make targetname KFLAGS=-DXTTPTYCMD (builds with xttptycmd())
1355 make targetname KFLAGS=-DYTTPTYCMD (builds with yttptycmd())
1356 make targetname KFLAGS=-DZTTPTYCMD (builds with zttptycmd())
1358 These are all in [95]ckutio.c. As noted, the second one works only for
1359 Linux, FreeBSD, NetBSD, and Mac OS X, because it uses non-POSIX,
1360 non-portable openpty(). If you want to try it on some other platform
1361 that has openpty(), you can build it like this:
1363 make targetname "KFLAGS=-DYTTPTYCMD -DHAVE_OPENPTY"
1365 (and let me know, so I can have HAVE_OPENPTY predefined for that
1366 platform too). The best strategy to get this working, I think, would be
1367 to concentrate on yttptycmd(), which is the simpler of the two
1368 pty-based routines. If it can be made to work, then we'll see if we can
1369 retrofit it to use the ckupty.c routines so it will be portable to
1372 By the way, if you build with any of [XYZ]TTPTYCMD defined, then the
1373 selected routine will always be used in place of ttruncmd(). This is to
1374 allow testing on all kinds of connections, not just secure ones, in
1375 both local and remote mode. Once the thing works, if it ever does, I'll
1376 add the appropriate tests and/or commands.
1378 By default, in the initial test release, C-Kermit 9.0 uses ttruncmd()
1379 on serial connections and ttyptycmd() on network connections. Even when
1380 a network connection is not encrypted, Kermit still needs to handle the
1381 network protocol, e.g. the quoting of 0xff bytes on Telnet connections.
1383 Demonstration: Fetch Mail from POP Server Secured by SSL
1385 [96]pop.ksc is a fully elaborated production script for fetching one's
1386 mail from a POP3 server over a connection secured by SSL. For
1387 explanation and documentation, [97]CLICK HERE. [98]mailcheck is a
1388 wrapper for the pop.ksc script, which collects your password one time,
1389 and then checks for new mail every 5 minutes (or other selected
1390 interval) and calls pop.ksc to fetch it if there is any.
1392 Demonstration: HP Switch Configuration Backup
1394 A common use for Kermit software is to make automated backups of the
1395 configuration of network switches and routers, such as those made by
1396 Cisco or Hewlett-Packard (although [99]tftp can be used for this, it is
1397 not available in all such devices; Kermit, however, works with those
1398 that have tftp as well as those that don't).
1400 Typically a backup can be done by making a Telnet, SSH, or serial
1401 connection to the device with Kermit and giving a command such as "show
1402 config" at the command-line prompt of the device with Kermit's session
1403 log activated. The result is a list of the commands that were used to
1404 establish the current configuration, suitable for feeding back to the
1405 device's console (e.g. with C-Kermit's TRANSMIT command) to reestablish
1406 the same configuration or to duplicate it on another device.
1408 At an HP installation it was noted, however, that while the HP switches
1409 (various ProCurve models) produced the desired list of commands, they
1410 were interspersed with escape sequences for special effects, thus
1411 rendering the recorded sessions unsuitable for feeding back into the
1414 C-Kermit 9.0 introduces a new feature to strip the offending sequences
1415 out of a session log, leaving just the text. The command SET
1416 SESSION-LOG TEXT activates this feature. In C-Kermit 9.0 Alpha.02 and
1417 earlier, escape sequence stripping occurred only while logging
1418 interactive (CONNECT) sessions; beginning with Alpha.03 it is done also
1419 for data that is read by INPUT commands and therefore works for scripts
1422 A sample HP Switch Configuration Backup script is [100]HERE, and its
1423 data file is [101]HERE. This script also illustrates some other new
1424 features of Alpha.03:
1427 This lets you put debugging messages in your script that can be
1428 displayed or not, according to SET DEBUG MESSAGE (below). This
1429 way you don't have to change your script for debugging. Hint:
1430 In Unix, invoke the script like this:
1432 $ DEBUG=1 scriptname arg1 arg2...
1434 and then include the following command in your script:
1436 if defined \$(DEBUG) set debug message on
1439 Like MESSAGE but prints the text with no line terminator, so it
1440 can be continued by subsequent messages.
1442 SET DEBUG MESSAGE { ON, OFF, STDERR }
1443 ON means MESSAGE commands should print to standard output; OFF
1444 means they shouldn't print anything; STDERR means the messages
1445 should be printed to [102]stderr. DEBUG MESSAGE is OFF by
1446 default, i.e. unless you SET it to ON or STDERR.
1449 Executes the command if SET DEBUG MESSAGE is not OFF.
1451 The \v(lastcommand) variable
1452 This variable contains the previous command. You can use it in
1453 debugging and error message to show (for example) exactly what
1454 the command was that just failed, without having to make a copy
1457 set host somehost.somecompany.com
1458 if fail exit 1 "FATAL - \v(lastcommand)"
1460 which, if the SET HOST command fails, prints "FATAL - set host
1461 somehost.somecompany.com" and then exits with status 1 (which
1462 normally indicates failure).
1464 Demonstration: HP iLO Blade Configuration
1466 [103]THIS DOCUMENT describes a script in production use at Columbia
1467 University for configuring and deploying racks full of HP blade servers
1468 through their "integrated Lights Out" (iLO) management interface,
1469 bypassing the tedious and error-prone process of configuring the
1470 servers one by one through the vendor-provided point-and-click Web
1471 interface, which is ill-suited to configuring large numbers of blades.
1472 The script illustrates some of C-Kermit 9.0's new features; source code
1473 is available through the link. The code is apt to change from time to
1474 time as new requirements surface.
1476 Demonstration: IBM/Rolm/Siemens CBX Management
1478 [104]THIS DOCUMENT describes a suite of scripts (some in production,
1479 some in development) used to manage the Columbia campus 20,000-line
1480 main telephone switch, along with about 10 satellite switches at
1481 off-campus locations. These switches are 1980s technology*, their
1482 management consoles are serial ports. Access is via Telnet to reverse
1483 terminal servers. The scripts allow for interactive sessions as well as
1484 automatic production (and in some cases formatting) of different
1485 reports required by different groups at different intervals. These
1486 scripts replace a whole assortment of ad-hoc ProComm ASPECT scripts
1487 that were scattered all over the place, with passwords embedded. The
1488 new scripts are intended to be run from a centralized server where
1489 there is a single well-secured configuration file, and where they can
1490 be used on demand, or in cron jobs. They are modular so code
1491 duplication is minimal.
1492 __________________________
1493 * Of course the University is deploying new technology but the but the
1494 old system will be used in parallel for some time to come.
1496 Demonstration: CSV and TSV Files
1500 * [105]Reading a CSV or TSV Record and Converting it to an Array
1501 * [106]Using \fjoin() to create a Comma- or Tab-Separated Value List
1503 * [107]Using CSV or TSV Files
1505 Comma-Separated Value (CSV) format is commonly output by spreadsheets
1506 and databases when exporting data into plain-text files for import into
1507 other applications. Here are the details:
1509 Comma-Separated List Syntax
1511 1. Each record is a series of fields.
1512 2. Records are in whatever format is used by the underlying file
1513 system for lines of text.
1514 3. Fields within records are separated by commas, with zero or more
1515 whitespace characters (space or tab) before and/or after the comma;
1516 such whitespace is considered part of the separator.
1517 4. Fields with embedded commas must be enclosed in ASCII doublequote
1519 5. Fields with leading or trailing spaces must be enclosed in ASCII
1521 6. Any field may be enclosed in ASCII doublequotes.
1522 7. Fields with embedded doublequotes must be enclosed in doublequotes
1523 and each interior doublequote is doubled.
1527 aaa, bbb, has spaces,,"ddd,eee,fff", " has spaces ","Muhammad ""The Greatest"" A
1530 The first two are regular fields. The second is a field that has an
1531 embedded space but in which any leading or trailing spaces are to be
1532 ignored. The fourth is an empty field, but still a field. The fifth is
1533 a field that contains embedded commas. The sixth has leading and
1534 trailing spaces. The last field has embedded quotation marks.
1536 Prior to C-Kermit 9.0 Alpha.06, C-Kermit did not handle CSV files
1537 according to the specification above. Most seriously, there was no
1538 provision for a separator to be surrounded by whitespace that was to be
1539 considered part of the separator. Also there was no provision for
1540 quoting doublequotes inside of a quoted string.
1542 Reading a CSV record
1544 Now the \fsplit() function can handle any CSV-format string if you
1545 include the symbolic include set "CSV" as the 4th parameter. To
1546 illustrate, this program:
1549 echo [\fcontents(\%1)]
1550 .\%9 := \fsplit(\fcontents(\%1), &a, \44, CSV)
1551 for \%i 1 \%9 1 { echo "\flpad(\%i,3). [\&a[\%i]]" }
1556 xx { aaa,,ccc," with spaces ",zzz }
1557 xx { "1","2","3","","5" }
1558 xx { this is a single field }
1559 xx { this is one field, " and this is another " }
1560 xx { name,"Mohammad ""The Greatest"" Ali", age, 67 }
1561 xx { """field enclosed in doublequotes""" }
1564 gives the following results:
1576 [ aaa,,ccc," with spaces ",zzz ]
1583 [ "1","2","3","","5" ]
1590 [ this is a single field ]
1591 1. [this is a single field]
1593 [ this is one field, " and this is another " ]
1594 1. [this is one field]
1595 2. [ and this is another ]
1597 [ name,"Mohammad ""The Greatest"" Ali", age, 67 ]
1599 2. [Mohammad "The Greatest" Ali]
1603 [ """field enclosed in doublequotes""" ]
1604 1. ["field enclosed in doublequotes"]
1607 The separator \44 (comma) must still be specified as the break set (3rd
1608 \fsplit() parameter). When "CSV" is specified as the include set:
1609 * The Grouping Mask is automatically set to 1 (which specifies that
1610 the ASCII doublequote character (") is used for grouping;
1611 * The Separator Flag is automatically set to 1 so that adjacent field
1612 separators will not be collapsed;
1613 * All bytes (values 0 through 255) other than the break character are
1614 added to the include set;
1615 * Any leading whitespace is stripped from the first element unless it
1616 is enclosed in doublequotes;
1617 * Any trailing whitespace is trimmed from the end of the last element
1618 unless it is enclosed in doublequotes;
1619 * If the separator character has any spaces or tabs preceding it or
1620 following it, they are ignored and discarded;
1621 * The separator character is treated as an ordinary data character if
1622 it appears in a quoted field;
1623 * A sequence of two doublequote characters ("") within a quoted field
1624 is converted to a single doublequote.
1626 There is also a new TSV symbolic include set, which is like CSV except
1627 without the quoting rules or the stripping of whitespace around the
1628 separator because, by definition, TSV fields do not contain tabs.
1630 Of course you can specify any separator(s) you want with either the
1631 CSV, TSV, or ALL symbolic include sets. For example, if you have a TSV
1632 file in which you want the spaces around each Tab to be discarded, you
1635 \fsplit(variable, &a, \9, CSV)
1639 The new symbolic include sets can also be used with \fword(), which is
1640 just like \fsplit() except that it retrieves the nth word from the
1641 argument string, rather than an array of all the words. In C-Kermit you
1642 can get information about these or any other functions with the HELP
1643 FUNCTION command, e.g.:
1645 C-Kermit> help func word
1647 Function \fword(s1,n1,s2,s3,n2,n3) - Extracts a word from a string.
1649 n1 = word number (1-based) counting from left; if negative, from right.
1650 s2 = optional break set.
1651 s3 = optional include set (or ALL, CSV, or TSV).
1652 n2 = optional grouping mask.
1653 n3 = optional separator flag:
1654 0 = collapse adjacent separators;
1655 1 = don't collapse adjacent separators.
1657 \fword() returns the n1th "word" of the string s1, according to the
1658 criteria specified by the other parameters.
1660 The BREAK SET is the set of all characters that separate words. The
1661 default break set is all characters except ASCII letters and digits.
1662 ASCII (C0) control characters are treated as break characters by default,
1663 as are spacing and punctuation characters, brackets, and so on, and
1664 all 8-bit characters.
1666 The INCLUDE SET is the set of characters that are to be treated as
1667 parts of words even though they normally would be separators. The
1668 default include set is empty. Three special symbolic include sets are
1671 ALL (meaning include all bytes that are not in the break set)
1672 CSV (special treatment for Comma-Separated-Value records)
1673 TSV (special treatment for Tab-Separated-Value records)
1675 For operating on 8-bit character sets, the include set should be ALL.
1677 If the GROUPING MASK is given and is nonzero, words can be grouped by
1678 quotes or brackets selected by the sum of the following:
1680 1 = doublequotes: "a b c"
1682 4 = apostrophes: 'a b c'
1683 8 = parentheses: (a b c)
1684 16 = square brackets: [a b c]
1685 32 = angle brackets: <a b c>
1687 Nesting is possible with {}()[]<> but not with quotes or apostrophes.
1690 Word number n1, if there is one, otherwise an empty string.
1697 Using \fjoin() to create Comma- or Tab-Separated Value Lists from Arrays
1699 In C-Kermit 9.0, \fsplit()'s inverse function, [108]\fjoin() received
1700 the capability of converting an array into a comma-separated or a
1701 tab-separated value list. Thus, given a CSV, if you split it into an
1702 array with \fsplit() and then join the array with \fjoin(), giving each
1703 function the new CSV parameter in the appropriate argument position,
1704 the result will be will be equivalent to the original, according to the
1705 CSV definition. It might not be identical, because if the result had
1706 extraneous spaces before or after the separating commas, these are
1707 discarded, but that does not affect the elements themselves. The new
1708 syntax for \fjoin() is:
1711 Given the array \&a[] or any other valid array designator, joins
1712 its elements into a comma-separated list according to the
1713 [109]rules listed above.
1716 Joins the elements of the given array into a tab-separated list,
1717 also described above.
1719 [110]Previous calling conventions for \fjoin() are undisturbed,
1720 including the ability to specify a portion of an array, rather than the
1723 declare \&a[] = 1 2 3 4 5 6 7 8 9
1724 echo \fjoin(&a[3:7],CSV)
1727 Using \fsplit() and \fjoin() it is now possible to convert a
1728 comma-separated value list into a tab-separated value list, and vice
1729 versa (which is not a simple matter of changing commas to tabs or vice
1732 Applications for CSV Files
1734 Databases such as MS Access or MySQL can export tables or reports in
1735 CSV format, and then Kermit can read the resulting CSV file and do
1736 whatever you like with it; typically something that could not be done
1737 with the database query language itself (or that you didn't know how to
1738 do that way): create reports or datasets based on complex criteria or
1739 procedures, edit or modify some fields, etc, and then use \fjoin() to
1740 put each record back in CSV form so it can be reimported into a
1741 spreadsheet or database.
1743 Here is a simple example in which we purge all records of customers who
1744 have two or more unpaid bills. The file is sorted so that each license
1745 purchase record is followed by its annual maintenance payment records
1746 in chronological order.
1748 #!/usr/local/bin/kermit
1749 .filename = somefile.csv # Input file in CSV format
1750 fopen /read \%c \m(filename) # Open it
1751 if fail exit # Don't go on if open failed
1752 copy \m(filename) ./new # Make a copy of the file
1754 .oldserial = 00000000000 # Multiple records for each serial number
1755 .zeros = 0 # Unpaid bill counter
1758 fread /line \%c line # Get a record
1759 if fail exit # End of file
1760 .n := \fsplit(\m(line),&a,\44,CSV) # Split the fields into an array
1761 if not equ "\m(oldserial)" "\&a[6]" { # Have new serial number?
1762 # Remove all records for previous serial number
1763 # if two or more bills were not paid...
1765 grep /nomatch \m(oldserial) /output:./new2 ./new
1768 .oldserial := \&a[6] # To detect next time serial number changes
1769 .zeros = 0 # Reset unpaid bill counter
1771 if equ "\&a[5]" "$0.00" { # Element 5 is amount paid
1772 increment zeros # If it's zero, count it.
1777 Rewriting the file multiple times is inelegant, but this is a quick and
1778 dirty use-once-and-discard script, so elegance doesn't count. The
1779 example is interesting in that it purges certain records based on the
1780 contents of other records. Maybe there is a way to do this directly
1781 with SQL, but why use SQL when you can use Kermit?
1783 Here is the same task but this time no shelling out, and this time we
1784 do change and add some fields and then join the result back into a CSV
1785 record and write it out to a new file. The object is to create a record
1786 for each license that shows not only the date and purchase price of the
1787 license but also the date and amount of the last maintenance payment,
1788 and to add new fields for sorting by anniversary (month and day):
1790 #!usr/local/bin/kermit +
1791 cd ~/somedirectory # CD to appropriate directory
1792 if fail exit 1 # Make sure we did
1793 .filename := \%1 # Filename from command line
1794 if not def filename { # If none give usage message
1795 exit 1 "Usage: \%0: infile [ outfile ]"
1797 fopen /read \%c \m(filename) # Open the input CSV file
1798 if fail exit # Make sure we did
1800 .output := \%2 # Output filename from command line
1801 if not def output { # Supply one if not given
1802 .output := New_\m(filename)
1804 fopen /write \%o \m(output) # Open output file
1805 if fail exit # Check that we did
1807 .serial = 00000000000 # Initialize serial number
1808 .licenses = 0 # and license counter
1810 fread /line \%c line # First line is column labels
1811 if fail exit # Check
1812 fwrite /line \%o "\m(line),AMM_DD,AYYYY" # Write new labels line
1814 # Remaining lines are license purchases (K95B) followed by zero or more
1815 # maintenance invoices (K95BM) for each license.
1817 .datepaid = 00/00/0000 # Initialize last maint payment date
1818 .amtpaid = $0.00 # Initialize last maint payment amount
1819 set flag off # For remembering we're at end of file
1820 while not flag { # Loop to read all records
1821 fread /line \%c line # Read a record
1822 if fail set flag on # If EOF set flag for later
1823 .n := \fsplit(\m(line),&a,\44,CSV) # Break record into array
1824 if ( flag || equ "\&a[3]" "K95B" ) { # License or EOF
1825 if fail exit 1 "FAILED: \v(lastcommand)"
1826 if licenses { # If this is not the first license
1827 .\&x[5] := \m(amtpaid) # Substitute most recent amount paid
1828 .\&x[21] := \m(datepaid) # Substitute most recent date paid
1829 void \fsplit(\&x[18],&d,/) # Break up original (anniversary) date
1830 # and put mm_dd and yyyy in separate fields for sorting...
1831 fwrite /line \%o "\fjoin(&x,CSV),\flpad(\&d[1],2,0)_\flpad(\&d[2],2,
1833 if fail exit 1 WRITE # Check for error
1834 xecho . # Show progress as one dot per record
1836 if flag break # We're at EOF so we're finished
1837 increment licenses # New license - count it
1838 array copy &a &x # Keep this record while reading next
1839 .serial := \&a[6] # Remember serial number
1840 .datepaid = 00/00/0000 # Initial maintenance payment date
1841 .amtpaid = $0.00 # and amount
1842 continue # and go back to read next record
1844 if not eq "\m(serial)" "\&a[6]" { # Catch out-of-sequence record
1846 echo "SEQUENCE: \m(serial)..\&a[6]: \&a[7] [\&a[1]]"
1849 if equ "\&a[5]" "" .\&a[5] = $0.00 # If amount is empty make it $0.00
1850 if not equ "\&a[5]" "$0.00" { # If amount is not $0.00
1851 .datepaid := \&a[21] # remember date paid
1852 .amtpaid := \&a[5] # and amount paid
1855 fclose ALL # Done - close all files and exit
1859 The result imports back into Excel, where it can be sorted, formatted,
1860 or otherwise manipulated as desired.
1862 Using CSV Files: Extending Kermit's Data Structures
1864 Now that we can parse a CSV record, what would we do with a CSV file -
1865 that is, a sequence of records? If we needed all the data available at
1866 once, we would want to load it into a matrix of (row,column) values.
1867 But Kermit doesn't have matrices. Or does it?
1869 Kermit has several built-in data types, but you can invent your own
1870 data types as needed using Kermit's macro feature:
1872 define variablename value
1876 define alphabet abcdefghijklmnopqrstuvwxyz
1878 This defines a macro named alphabet and gives it the value
1879 abcdefghijklmnopqrstuvwxyz. A more convenient notation (added in
1880 C-Kermit 7.0, see [111]Table 2) for this is:
1882 .alphabet = abcdefghijklmnopqrstuvwxyz
1884 The two are exactly equivalent: they make a literal copy the "right
1885 hand side" as the value of the macro. Then you can refer to the macro
1886 anywhere in a Kermit command as "\m(macroname)":
1888 echo "Alphabet = \m(alphabet)"
1890 There is a second way to define a macro, which is like the first except
1891 that the right-hand side is evaluated first; that is, any variable
1892 references or function calls in the right-hand side are replaced by
1893 their values before the result is assigned to the macro. The command
1894 for this is ASSIGN rather than DEFINE:
1896 define alphabet abcdefghijklmnopqrstuvwxyz
1897 assign backwards \freverse(\m(alphabet))
1898 echo "Alphabet backwards = \m(backwards)"
1902 Alphabet backwards = zyxwvutsrqponmlkjihgfedcba
1904 This kind of assignment can also be done like this:
1906 .alphabet = abcdefghijklmnopqrstuvwxyz
1907 .backwards := \freverse(\m(alphabet))
1909 [112]Any command starting with a period is an assignment, and the
1910 operator (= or :=) tells what to do with the right-hand side before
1911 making the assignment.
1913 In both the DEFINE and ASSIGN commands, the variable name itself is
1914 taken literally. It is also possible, however, to have Kermit compute
1915 the variable name. This is done (as described in [113]Using C-Kermit,
1916 2nd Ed., p.457), using parallel commands that start with underscore:
1917 _DEFINE and _ASSIGN (alias _DEF and _ASG). These are just like DEFINE
1918 and ASSIGN except they evaluate the variable name before making the
1919 assignment. For example:
1922 _define \%a\%a\%a 111
1924 would create a macro named ONEONEONE with a value of 111, and:
1928 _assign \%a\%a\%a \m(number)
1930 would create the same macro with the same value, but:
1934 _define \%a\%a\%a \m(number)
1936 would give the macro a value of "\m(number)".
1938 You can use the _ASSIGN command to create any kind of data structure
1939 you want; you can find some examples in the [114]Object-Oriented
1940 Programming section of the [115]Kermit Script Library. In the following
1941 program we use this capability to create a two-dimensional array, or
1942 matrix, to hold the all the elements of the CSV file, and then to
1945 fopen /read \%c data.csv # Open CSV file
1949 .\%m = 0 # Maximum columns
1951 fread /line \%c line # Read a record
1952 if fail break # End of file
1953 .\%n := \fsplit(\m(line),&a,\44,CSV) # Split record into items
1954 incr \%r # Count this row
1955 for \%i 1 \%n 1 { # Assign items to this row of matrix
1956 _asg a[\%r][\%i] \&a[\%i]
1958 if > \%i \%m { .\%m := \%i } # Remember width of widest row
1960 fclose \%c # Close CSV file
1961 decrement \%m # (because of how FOR loop works)
1962 echo MATRIX A ROWS: \%r COLUMNS: \%m # Show the matrix
1964 for \%i 1 \%r 1 { # Loop through rows
1965 for \%j 1 \%m 1 { # Loop through columns of each row
1966 xecho "\flpad(\m(a[\%i][\%j]),6)"
1972 The matrix is called a and its elements are a[1][1], a[1][2], a[1][3],
1973 ... a[2][1], etc, and you can treat this data structure exactly like a
1974 two-dimensional array, in which you can refer to any element by its "X
1975 and Y coordinates". For example, if the CSV file contained numeric data
1976 you could compute row and column sums using simple FOR loops and
1977 Kermit's built-in one-dimensional array data type:
1979 declare \&r[\%r] # Make an array for the row sums
1980 declare \&c[\%m] # Make an array for the column sums
1981 for \%i 1 \%r 1 { # Loop through rows
1982 for \%j 1 \%m 1 { # Loop through columns of each row
1983 increment \&r[\%i] \m(a[\%i][\%j]) # Accumulate row sum
1984 increment \&c[\%j] \m(a[\%i][\%j]) # Accumulate column sum
1988 Note that the sum arrays don't have to be initialized to zero because
1989 Kermit's INCREMENT command treats empty definitions as zero.
1991 Demonstration Scripts for Webmasters
1993 These scripts all use new features of C-Kermit 9.0.
1996 A C-Kermit 9.0 script to build sitemap.xml for a website,
1997 complete with Google image extensions (this is the file used by
1998 webmasters to get their sites crawled and indexed optimally).
2000 [117]The Weblog Script
2001 Reads a web log, extracts the Google searches, normalizes the
2002 search strings, and prints the top 20 searches, along with their
2005 [118]The Amazon Script
2006 Reads an Amazon Associate orders report and lists the products
2007 according to the number of orders for each, or the number of
2011 Makes a website from a collection of JPG images.
2013 [120]Home [121]Kermit 95 [122]C-Kermit [123]Scripts [124]Current
2014 [125]New [126]FAQ [127]Support
2017 C-Kermit 9.0 / [128]The Kermit Project / [129]Columbia University /
2018 [130]kermit@columbia.edu / [131]validate
2022 1. http://www.columbia.edu/
2023 2. mailto:kermit@columbia.edu
2024 3. http://www.columbia.edu/kermit/index.html
2025 4. http://www.columbia.edu/kermit/k95.html
2026 5. http://www.columbia.edu/kermit/ckermit.html
2027 6. http://www.columbia.edu/kermit/ckscripts.html
2028 7. http://www.columbia.edu/kermit/current.html
2029 8. http://www.columbia.edu/kermit/whatsnew.html
2030 9. http://www.columbia.edu/kermit/faq.html
2031 10. http://www.columbia.edu/kermit/support.html
2032 11. http://www.columbia.edu/cu/computinghistory/books/#menagerie
2033 12. http://www.columbia.edu/kermit/ck90tables.html
2034 13. http://www.amazon.com/gp/product/1555581641?ie=UTF8&tag=aleidmoreldom-20&linkCode=as2&camp=1789&creative=9325&creativeASIN=1555581641
2035 14. http://www.columbia.edu/kermit/ckermit.html#download
2036 15. http://www.columbia.edu/kermit/ckermit90.html#LargeFiles
2037 16. http://www.columbia.edu/kermit/ckermit90.html#TestLargeFiles
2038 17. http://www.columbia.edu/kermit/ckermit90.html#Bignums
2039 18. http://www.columbia.edu/kermit/ckermit90.html#force3
2040 19. http://www.columbia.edu/kermit/ckermit90.html#Vareval
2041 20. http://www.columbia.edu/kermit/ckermit90.html#rename
2042 21. http://www.columbia.edu/kermit/ckermit90.html#Other
2043 22. http://www.columbia.edu/kermit/ckermit90.html#Incompatibilities
2044 23. http://www.columbia.edu/kermit/ckermit90.html#NotIn9.0
2045 24. http://www.columbia.edu/kermit/ckermit90.html#LooseEnd
2046 25. http://www.columbia.edu/kermit/ckermit90.html#pop
2047 26. http://www.columbia.edu/kermit/ckermit90.html#HPswitch
2048 27. http://www.columbia.edu/kermit/ckermit90.html#iLO
2049 28. http://www.columbia.edu/kermit/ckermit90.html#Rolm
2050 29. http://www.columbia.edu/kermit/ckermit90.html#CSV
2051 30. http://www.columbia.edu/kermit/ckermit90.html#Otherdemos
2052 31. http://www.columbia.edu/kermit/ck60manual.html
2053 32. http://www.amazon.com/gp/product/B002ACPF9M?ie=UTF8&tag=aleidmoreldom-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=B002ACPF9M
2054 33. http://www.columbia.edu/kermit/ckermit70.html
2055 34. http://www.columbia.edu/kermit/ckermit80.html
2056 35. http://www.columbia.edu/kermit/ckscripts.html
2057 36. http://www.columbia.edu/cu/computinghistory/dec20.html
2058 37. mailto:fdc@columbia.edu
2059 38. http://www.columbia.edu/kermit/k95.html
2060 39. http://www.columbia.edu/kermit/cu-bsd-license.html
2061 40. http://www.columbia.edu/kermit/ckermit90.html#LargeFiles
2062 41. http://www.columbia.edu/kermit/ck90tables.html
2063 42. http://www.columbia.edu/kermit/ck90tables.html
2064 43. http://www.columbia.edu/kermit/ckermit90.html#force3
2065 44. http://www.columbia.edu/kermit/ckermit90.html#Vareval
2066 45. http://www.columbia.edu/kermit/ckrename.html
2067 46. http://www.columbia.edu/kermit/csv.html
2068 47. http://www.columbia.edu/kermit/csetnames.html
2069 48. http://www.columbia.edu/kermit/ckermit90.html#HPswitch
2070 49. http://www.columbia.edu/kermit/ckdaily.html
2071 50. http://www.columbia.edu/kermit/cu-bsd-license.html
2072 51. http://www.opensource.org/
2073 52. http://kermit.columbia.edu/ck90tables.html#LF
2074 53. ftp://kermit.columbia.edu/kermit/utils/bigfile.c
2075 54. http://www.columbia.edu/kermit/ckermit80.html#x9
2076 55. http://www.columbia.edu/kermit/ck90tables.html#LF
2077 56. ftp://kermit.columbia.edu/kermit/scripts/ckermit/easter2
2078 57. http://www.columbia.edu/kermit/em-apex.html
2079 58. http://www.iridium.com/
2080 59. http://science1.nasa.gov/science-news/science-at-nasa/2006/09jan_electrichurricanes/
2081 60. http://www.columbia.edu/kermit/ek.html
2082 61. ftp://kermit.columbia.edu/kermit/ek/simirid/
2083 62. http://www.columbia.edu/kermit/ek.html
2084 63. http://www.columbia.edu/kermit/ckermit70.html#x7.10.10
2085 64. http://www.columbia.edu/kermit/csv.html
2086 65. http://www.columbia.edu/kermit/ckermit70.html#x1.11
2087 66. http://www.columbia.edu/kermit/ckermit70.html
2088 67. http://www.columbia.edu/kermit/ckermit80.html#x9
2089 68. http://www.columbia.edu/kermit/csetnames.html
2090 69. http://www.columbia.edu/kermit/ckdaily.html
2091 70. http://www.columbia.edu/kermit/ftpclient.html
2092 71. http://www.columbia.edu/kermit/ckdaily.html
2093 72. http://www.columbia.edu/kermit/csetnames.html
2094 73. http://www.columbia.edu/kermit/ckermit90.html#Otherdemos
2095 74. http://www.columbia.edu/kermit/input_nowrap.html
2096 75. http://www.columbia.edu/~fdc/mm/index.html
2097 76. http://www.columbia.edu/kermit/photoalbum.html
2098 77. http://www.columbia.edu/~fdc/kermit/logserial.html
2099 78. http://www.columbia.edu/kermit/ckermit90.html#message
2100 79. http://www.columbia.edu/kermit/ckermit90.html#NotIn9.0
2101 80. http://www.columbia.edu/kermit/csv.html#join
2102 81. http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2009-3767
2103 82. http://www.columbia.edu/kermit/ftp/test/tar/x.zip
2104 83. http://www.mseclab.com/?p=180
2105 84. http://www.theregister.co.uk/2009/07/30/universal_ssl_certificate/
2106 85. http://www.columbia.edu/kermit/ckermit90.html#HPswitch
2107 86. http://www.columbia.edu/kermit/em-apex.html
2108 87. http://www.freebsd.org/releases/8.0R/announce.html
2109 88. http://www.columbia.edu/kermit/security81.html#x4.2.3
2110 89. http://www.columbia.edu/kermit/ckermit80.html#x9
2111 90. http://www.columbia.edu/kermit/ckermit80.html#x8.7.2
2112 91. http://www.columbia.edu/kermit/ckermit80.html#x9
2113 92. http://www.columbia.edu/kermit/ckpackages.html
2114 93. http://docs.hp.com/en/B9106-90013/pty.7.html
2115 94. http://www.columbia.edu/kermit/test/text/ckc299.txt
2116 95. http://www.columbia.edu/kermit/test/text/ckutio.c
2117 96. http://www.columbia.edu/~fdc/mm/pop
2118 97. http://www.columbia.edu/~fdc/mm/
2119 98. http://www.columbia.edu/~fdc/mm/mailcheck
2120 99. http://en.wikipedia.org/wiki/Trivial_File_Transfer_Protocol
2121 100. http://www.columbia.edu/kermit/ftp/scripts/ckermit/gethpconfig
2122 101. http://www.columbia.edu/kermit/ftp/scripts/ckermit/TestSwitches.txt
2123 102. http://en.wikipedia.org/wiki/Standard_streams
2124 103. http://kermit.columbia.edu/cudocs/ilosetup.html
2125 104. http://www.columbia.edu/kermit/cudocs/cbx.html
2126 105. http://www.columbia.edu/kermit/ckermit90.html#record
2127 106. http://www.columbia.edu/kermit/ckermit90.html#join
2128 107. http://www.columbia.edu/kermit/ckermit90.html#file
2129 108. http://www.columbia.edu/kermit/ckermit80.html#fjoin
2130 109. http://www.columbia.edu/kermit/ckermit90.html#rules
2131 110. http://www.columbia.edu/kermit/ckermit80.html#fjoin
2132 111. http://www.columbia.edu/kermit/ckermit90.html#varasg
2133 112. http://www.columbia.edu/kermit/ckermit70.html#x7.9
2134 113. http://www.amazon.com/gp/product/1555581641?ie=UTF8&tag=aleidmoreldom-20&linkCode=as2&camp=1789&creative=9325&creativeASIN=1555581641
2135 114. http://www.columbia.edu/kermit/ckscripts.html#oops
2136 115. http://www.columbia.edu/kermit/ckscripts.html
2137 116. http://www.columbia.edu/kermit/ksitemap.html
2138 117. http://www.columbia.edu/kermit/weblog.html
2139 118. http://kermit.columbia.edu/ftp/scripts/ckermit/amazon
2140 119. http://www.columbia.edu/kermit/photoalbum.html
2141 120. http://www.columbia.edu/kermit/index.html
2142 121. http://www.columbia.edu/kermit/k95.html
2143 122. http://www.columbia.edu/kermit/ckermit.html
2144 123. http://www.columbia.edu/kermit/ckscripts.html
2145 124. http://www.columbia.edu/kermit/current.html
2146 125. http://www.columbia.edu/kermit/whatsnew.html
2147 126. http://www.columbia.edu/kermit/faq.html
2148 127. http://www.columbia.edu/kermit/support.html
2149 128. http://www.columbia.edu/kermit/index.html
2150 129. http://www.columbia.edu/
2151 130. mailto:kermit@columbia.edu
2152 131. http://validator.w3.org/check?uri=http%3A%2F%2Fkermit.columbia.edu%2Fckermit90.html
projects / ckermit.git / blob