Documentation for UUENCODE/DECODE 5.22 UU-encoding is a way to code a file which may contain any characters into a standard character set that can be reliably sent over diverse networks. THE CHARACTER ENCODING: The basic scheme is to break groups of 3 eight bit characters (24 bits) into 4 six bit characters and then add 32 (a space) to each six bit character which maps it into the readily transmittable character. Another way of phrasing this is to say that the encoded 6 bit characters are mapped into the set: `!"#$%&'()*+,-./012356789:;<=>?@ABC...XYZ[\]^_ for transmission over communications lines. As some transmission mechanisms compress or remove spaces, spaces are changed into back-quote characters (a 96). (A better scheme might be to use a bias of 33 so the space is not created, but this is not done.) Another newer less popular encoding method, called XX-encoding uses the set: +-01..89ABC...XYZabc...xyz In my opinion, XX-encoding is superior to UU-encoding because it uses more "normal" characters that are less likely to get corrupted. In fact several of the special characters in the UU set do not get thru an EBCDIC to ASCII translation correctly. Conversely, an advantage of the UU set is that it does not use lower case characters. Now-a-days both upper and lower case are sent with no problems; maybe in the communications dark ages, there was a problem with lower case. This "UU" encode/decode pair can handle either XX or UU encoding. The encode program defaults to creating a UU encoded file; but can be run with a "-x" option to create an XX encoding. The decode program defaults to autodetect. However the program can get confused by comment lines preceeding the actual encoded data. The decode mode can be forced to UU or XX with the "-u" or "-x" parameter. Another option is for the character mapping table to be inserted at the front of the file. The format for this is discussed later. The table parameters are detected and used by this decode program. (A table will override the "-x" or "-u" parameters.) The encode program can be run with a "-t" option which tells it to put the table into the encoded file. A third encode mapping is the one used by Brad Templeton's ABE program. This is not handled by these programs as the check and control information surrounding the actual encoded data is in a different form. From a theoritical view, this encoding is breaking down 24 bits modulo 64. Note that 64**3 is = 2**24. The result is 24 bits in for 32 bits out, a 33% size increase. Note that 85**5 > 2**32. Also note that there are 94 transmittable ASCII characters (from 0x21 thru 0x7e). Thus modulo 85 encoding (the atob encoder) transforms 32 bits to 5 ASCII chars or 40 bits for a 25% size increase. The trade off in the module 85 encoding is that many communications systems do not reliably transmit 85 ASCII characters. The tilda, carat, brackets, and sometimes upper or lower case frequently get corrupted. COMPOSING A LINE OF ENCODED CHARACTERS: A small number of eight bit characters are encoded into a single line and a count is put at the start of the line. (Most lines in an encoded file have 45 encoded characters. When you look at a UU-encoded file note that most lines start with the letter "M". "M" is decimal 77 which, minus the 32 bias, is 45.) This encode program puts a check character at the end of each line. The check is the sum of all the encoded characters, before adding the mapping, modulo 64. Note: Horton 9/1/87 UUENCODE has a bug in the line check algorithm; it uses the sum of the original, not the encoded characters. This decode program accepts either form of line check character. In previous versions (4.13 and lower) the line check characters was generated by default by this encode program and was supressed with the "-L" option. One reason to supress them is if they will be decoded by one of the old Horton decoders. Most decoders either accept this form of check or simply stop looking after the line length is exhausted. My feelings are mixed about the line checksums because errors of this type essentially never occur. However with modern, error-free communications systems and with the CRC checks on the entire file (see below) I have made the default for uuencoding to have NO line level check characters effective version 4.21. The "-L" option on uuencode turns on generation of line checksums. If you have a really bad communications system and you want to isolate a problem, turn them on. Uudecode automatically checks for the presense line checksums, so the default for uudecode is to leave line level checks on; if there are some problems the "-L" option for uudecode turns them off. Sometimes there is junk at the end of the line which causes spurious line checksum errors. I have encountered various other ways that encoders end lines. One encoder put a "M" at both the start and end of the line. Another used a line count character. This decode program checks all of these. I would not be surprised if some encoder out there ends lines with astrological symbols. If you encounter some other wierd form of encoded file, let me know. PACKAGING THE LINES INTO FILES: The lines of encoded data can be preceded by comments and by network addressing information. The encoded data is directly preceded by a line containing: begin This line is created by the encoding program. The decode program scans the file looking for "begin" in column 1. The final end of encoded data is an encoded line with zero encoded characters (a back-quote), followed by a line containing "end". For integrity checking, various encode programs insert checksums for the entire file. This decode tries to check for all known types of file checksums. This is discussed in more detail later. This encode program puts a header line, containing the section number and file name, in front of every section: "section of uuencode of file " At the end of a section the encode program inserts a line containing checksum and file size information. This can be suppressed with the "-c" option. The transmissions on comp.binaries.ibm.pc contain two different types of section number/file-name lines. The first is the "Archive-name line"; the other is the "part line". The format of the Archive-name line is: "Archive-name: /part" for example: Archive-name: diskutl/part02 The format of the part line is: part/ for example: diskutl part02/03 This program checks for consistency of these names and numbers as of release 5.0. All the "integrity fields" (the checksum, the line check, and the section header line) are inserted in a way that they will be ignored by other UUDECODE programs that cannot handle them. This decode program does not require any of these fields; if not present, integrity checking is not done. This program pair is 100% downward compatible! FILE NAMES: The default name of the file to encode into is the base name of the file you are encoding plus the ".UUE" extension for UU encoding. uuencode foo.bar produces file foo.uue. If the -X (upper case) option is used, then the file will be XXencoded (see above); and will be saved with the default ".XXE" extension. Uuencode can also be called with a second parameter which is the specific file name of the encoded file. If this file name has no extension, the above default (.UUE or .XXE is used). uuencode foo foo.bar encode foo to foo.bar uuencode -X foo xxfoo encode foo to foo.xxe uuencode -X foo xxfoo.bar encode foo to xxfoo.bar uuencode foo xxfoo. encode foo to xxfoo. Uudecode defaults to look for files with the ".UUE" and the ".XXE" extension. This only applies if uudecode is called without a file name extension: uudecode foo look for file foo.uue, then foo.xxe uudecode -u foo look for file foo.uue only uudecode -x foo look for file foo.xxe, then foo.uue If uudecode is called with a file name extension, then that is used: uudecode foo.XXX decode file foo.xxx uudecode foo. decode file foo with no extension SPLITING UP LONG FILES: Long files are broken into several sections before transmission. This is done because very large files are cumbersome to handle and because some networks require files to be less than 64K bytes. This encode program automatically breaks large encoded files into sections. This split is controlled by several options. First the "-s" option tells encode not to split the file. The "-s nnn" option tells encode to split the file into hunks of "nnn" lines. The default is 950 lines which is about 59k. Sometimes extensive comments are put into the first file; thus it may be necessary for the first file to contain fewer encoded lines. The "-h nnn" option tells encode to leave room for "nnn" additional lines in the first file. If the data file being encoded is called FOO.ZIP, then the encode program names the encoded files FOO1.UUE, FOO2.UUE, etc. (Or .XXE if the -X option is used.) The decode program searches for the various sections, scans over preliminary comments and decodes all as if they were one big file. Decode is passed the base file name "FOO". UUdecode can (but rarely does) get confused and thinks header lines are encoded data. Sometimes this is because the seperator line between sections (the "cut" line) is indistinguishable from valid decodable data. (An example is the line "---" used as a cut line on several DOS BBS systems.) You can tell UUdecode that a specific line is a cut line and not a decodable line with the -Z option: uudecode -Z "---" myfile Other times there is not cut line in between file sections or there is some other problem with the file. If so, edit the file and try again. This has only happened once to me and I have decoded a lot of files. When decode encounters a premature end-of-file or some data which is not decodable, it assumes the end of a file section. Decode is conservative when it encounters data it cannot decode (better an error than a bad file). Usually this undecodable data is valid "trailer" data put at the end of file for data transmission purposes. However the file may be bad. So decode continues to scan the file, if decode then encounters a line which is decodable it assumes the file is bad. Or if there are more than 30 lines remaining in the file, decode assumes the file is bad. When decode encounters a valid end of file section it must get the next file in sequence. If the file name ends with a number, decode tries the next file name in numeric sequence. Otherwise decode asks for a file name. If this file does not contain decodable data, decode asks for another file to try. If multiple sections are saved in a single file, the sections must have one of the three (above) types of section header lines for validation. Decode builds a table of section information so it can go back and reread if sections are not saved in order. The "SECTION" line inserted by the UUENCODE program is used for validity checking only. If not present, decode will accept any file containing encoded lines. OTHER FILE FORMS: Sometimes files are wrapped in shell archives that automatically check sequencing and call uudecode for you on Unix systems. If you prefer to download the raw files to MS-DOS, uudecode 5.22 will filter simple shell scripts, that use the Unix 'sed' command, and decode the data automatically. There is one more rarely used feature of ENCODE: many input files can be encoded into one large encode file. (I have never seen this used.) The end of an input file is a zero length encoded line, followed by another "begin" line instead of by an "end" line. This decode program will decode this sort of file; but the encode will only handle a single input file. FILE LEVEL CHECKSUMS: There are three types of file checksums found in encoded files: UUENCODE 2.14 and below inserted lines that gave the section size and the original input file size. This is supplanted by a better technique in 3.07; but 3.07 UUDECODE still checks and validates the old form UUENCODE 3.07 and Rahul Dhesi's encode scripts compute a Unix "sum -r" on the encoded sections and on the original input file. A difference is that UUENCODE 3.07 puts the expected "sum -r" and size at the end of a file while Rahul''s scripts put them at begining. This UUDECODE analyzes either. The third form of checksum is a full 32 bit CRC that Rahul's script inserts. My code does not handle this. Rahul has written the BRIK program to check them. If there is a "sum -r" failure, BRIK analysis should be considered. Unisys Unix platforms put a line containing just the original file size at the end of the encoded file. My code checks this. TABLE LINES: Some encoded files but the mapping used at the front of the encoded file, just in front of the "begin" line. The format for this is: table first 32 characters second 32 characters All this starts in column 1. If decode encounters a table specification, it uses it and overrides any command line parameters. Encode will create the table lines if run with the "-t" parameter. COMPLETION CODES: On successful completion, UUDECODE sets ERRORLEVEL to 0. If there are any problems, ERRORLEVEL is set to non-zero. Most of the options to UUDECODE are obvious. However, the "-e" option needs more explanation. The purpose of "-e" is to automatically run an un-archiver (like ZOO or PKUNPAK) when UUDECODE successfully completes. If the "-e" option is given, UUDECODE calls BAT file UNARCUUE on successful completion; UNARCUUE is passed five parameters: the filename decoded (with path but no extension), the file extension, the input file name (with path but no extension), the input file extension that is used, the number of sections. Normally the file extension tells which un-archiver to call. The UNARCUUE BAT file, can test these parameters and call the necessary un-archiver. If UNARCUUE is called, the return code from UUDECODE is the return code passed back from UNARCUUE. One user had a problem in that the routines called by UNARCUUE set the errorlevel to 1 which was passed back to be the return code from UUDECODE. This works well for me. On UNIX I find a program I want in three sections: PRG1, PRG2, PRG3. I copy the three files down to my PC as PRG1.UUE, PRG2.UUE, and PRG3.UUE. I then just enter UUDECODE PRG and the thing decodes. Done privately and not for profit (freeware). Suggestions appreciated. The programs are written in Turbo Pascal 5.5 with about 5% TASM for speed. The source is not public domain. I would entertain consulting contracts for porting to other hardware platforms. Also if included in your for profit product, please contact me. Richard Marks 931 Sulgrave Lane Bryn Mawr, PA 19010 Copyright Richard E. Marks, Bryn Mawr, PA, 1992 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Change log (started with 5.13): 5.22 Improve analysis of "part" lines to accept form used in bin.pictures group. Fixes a problem with encoded files that use blanks rather than back quotes. 5.20 Z command line option to specify a "cut" line between multiple sections. Needed if cut line is a valid decodable line (of low probability) which the user chooses to be intrepreted as a "cut" line. Plus other improvements in detecting end of section. 5.16 Encode will split to a minimum of 75 (was 150) lines. Passes more info to UNARCUUE 5.15 Fixes a problem with trailing blanks on lines. 5.14: Fixes a minor bug in which a redundant error message was produced when decoding single section files. 5.13 VERSUS 5.10: 5.13 decode has a command line option that disables all interactive responses to make it more useable from some BBS systems. Examine the "y" and "Y" options. 5.13 can increment the number on files up to five digits. The prior limit was two digits. You can now save files with names based on news article numbers. 5.13 can decode files encoded into 100 or more parts. A restriction is that if there are more than 100 parts, the sections MUST be in order.