Controlling the File Transfer
The current Secure Shell protocol and current clients do not transfer
any information about the files to be transferred, only the file
contents as a byte stream. This is sufficient for Unix-type files if the
sender and receiver use the same CCS.
SSH Tectia Server (M) needs more information: which transfer format to use, what
code sets are involved, and what the file characteristics are. Some of
this information may be transported in the protocol in the future versions
of SSH Tectia Client. In SSH Tectia Server (M), the information is either encoded in the filename
(Advice String), read from a file transfer profile, or from
environment variables.
Advice String
Since the SFTP server decodes the name for the dataset, the filename
in the request can be used to convey other information. The following
filename convention is used: any filename that starts with
/ftadv: has the format
/ftadv:advstr/realfilename
where advstr is the advice string and realfilename is
the dataset name or a filename to be further processed by the server.
The advice string is a sequence of name=value pairs delimited
by commas. The names are as follows:
-
X=transfer_mode The transfer mode specifies whether codeset and line delimiter conversions
are performed. Possible values are:
X=bin: Codeset and line delimiter conversions are not performed.
X=text: Codeset and line delimiter conversions are performed.
-
F=transfer_format The byte stream consists of the bytes that are transferred as payload in
the SFTP protocol packets. The byte stream has one of the following
formats: stream, line, or record. All three
formats may have data consisting of text, non-text data, or a mixture of
these. When creating an MVS dataset, a record that is longer than the
maximum or fixed record length for the dataset will be truncated. When
writing to datasets with fixed record lengths, short records will be
filled with binary zeroes if you use the record transfer format and with
blanks if you use the line transfer format.
F=stream: The stream transfer format contains the data bytes of
the dataset but no structural information. If a dataset with a fixed record
length is transferred with the stream format and recreated with the same
record length, the record structure will be preserved. Variable length
records will not be recreated properly if transferred with the stream format.
F=line: The line transfer format is record-based. It uses
delimiter characters to mark the end of a record. The delimiter character
may be a Carriage Return or a Newline. When writing to or reading from
datasets with ASA control characters, a Form Feed is also treated as a
delimiter. The table below shows the values of these characters in EBCDIC
and ASCII. Data sent to SSH Tectia Server (M) in the line transfer format must be
in EBCDIC or must be converted to EBCDIC during the transfer.
Delimiter EBCDIC ASCII
Name Dec Oct Hex Name Dec Oct Hex
\r Carriage Return CR 13 015 0x0D CR 13 015 0x0D
\n Newline NL 21 025 0x15 LF 10 012 0x0A
\f Form Feed FF 12 014 0x0C FF 12 014 0x0C
Note that ASCII does not have a NL character, instead LF is used to
delimit lines.
Avoid conversions that transform an ASCII Line Feed (LF/10/012/0x0A)
into an EBCDIC Line Feed (LF/37/045/0x25) or an EBCDIC Newline
(NL/21/025/0x15) into an ASCII Next Line (NEL/133/0205/0x85).
Be aware that sending a double delimiter, e.g. \r\n
or \n\r, to SSH Tectia Server (M) will result in two
records. The transfer-line-delimiter and
file-line-delimiter advice string attributes can be used to
cause the SSH Tectia Server (M) server or client program to convert between the
line delimiter conventions.
SSH Tectia Server (M) sends \n as the Server Newline Convention
in the server initialization SFTP protocol message.
When transferring line format data to and from MVS files with ASA line
printer control characters, SSH Tectia Server (M) will convert between the
control characters and line delimiter characters, as described in the
IBM book z/OS C/C++ Programming Guide, SC09-4765-03, Chapter
8.
Datasets transferred in the line transfer format and recreated on a
mainframe will not necessarily be identical.
F=record: The record transfer format is record-based. Each record
is preceded by a length field consisting of a 4-byte big-endian binary
integer, which indicates the number of data bytes in the record. Note that
the format is not the same as the record descriptor word in datasets with
RECFM=V or VB.
A dataset that is transferred with the record transfer format can be
recreated as any dataset type.
-
D=file_codeset The data in the dataset has the specified codeset. file_codeset
is the codeset name that is known to the iconv function of the
system performing the conversion. The available codesets can be listed by
invoking the iconv command at a USS prompt with the -l
option:
>iconv -l
-
C=transfer_codeset During the transfer the data has the specified codeset. transfer_codeset
is the codeset name that is known to the iconv function
of the system performing the conversion. The available codesets can be
listed by invoking the iconv command at a USS prompt with the
-l option:
>iconv -l
-
E=translate_table translate_table is the name of the table that specifies the codeset
conversion. If set, this attribute overrides the transfer codeset and file
codeset attributes. The table is always applied in the normal
direction, that is, the first character array is used for incoming
(from the line to the dataset) data and the second array for outgoing
data. If the opposite translation is needed, e.g. the dataset contains
ASCII and should be transferred as EBCDIC, the users (or their system
programmer) can prepare a table dataset with the character arrays in
reversed order (e.g. with the system utility CONVXLAT or by editing an
existing translate dataset).
-
A=translate_dsn_templates translate_dsn_templates specifies the search templates for the translate
table. The first match is used as the translate table.
-
J=file_line_delimiter The file line delimiter specifies the newline convention used in the
file. Possible values are:
J=mvs: The line delimiter used in the file is NL (\n, 0x0a).
J=unix: The line delimiter used in the file is NL (\n, 0x0a).
J=dos: The line delimiter used in the file is LFNL (\r\n, 0x0d0a).
J=mac: The line delimiter used in the file is LF (\r, 0x0d).
-
I=transfer_line_delimiter The transfer line delimiter specifies the newline convention used during
the file transfer. Possible values are:
I=mvs: The line delimiter during the transfer is NL (\n, 0x0a).
I=unix: The line delimiter during the transfer is NL (\n, 0x0a).
I=dos: The line delimiter during the transfer is LFNL (\r\n, 0x0d0a).
I=mac: The line delimiter during the transfer is LF (\r, 0x0d).
-
T=file_type The file type specifies the type of the dataset when the dataset is created.
T=hfs: The type of the created dataset is HFS.
T=po: The type of the created dataset is PDS.
T=poe: The type of the created dataset is PDSE.
T=vsam: The type of the created dataset is VSAM.
T=ps: The type of the created dataset is PS.
-
O=file_org file_org specifies the dataset organization. Possible values are all
valid combinations of the following letters:
F Fixed
V Variable
U Undefined
B Blocked
S Spanned or standard
M Machine line printer codes
A ASA line printer codes
-
R=file_reclen Maximum record length or fixed record length.
-
B=file_blocklen Maximum block size.
-
L=file_size Size estimate for dataset allocation.
-
M=directory_size Number of 256-byte records in the directory.
-
P=file_transfer_profile The file transfer profile specifies the name profile used for the file
transfer. The profile name is case-sensitive.
Example Advice Strings
Below is an example of a filename that requests the transfer of a PDS
member with the line transfer format and codeset conversion from
EBCDIC to an ASCII codeset:
/ftadv:D=IBM-1047,C=ISO8859-1,F=line/__personal.cntl/idlist
Different combinations of underscore and slash (__, _/,
/_ or //) in front of the filename indicate that the
file is an MVS dataset.
This example names an HFS file to be transferred without changes:
/ftadv:X=bin,T=HFS,F=stream/profcopy
In the example, the transfer mode is set to binary (X=bin) to
avoid conversion and to override any defaults set in the matching file
transfer profiles or environment files.
The next example uses name file transfer profile myprofile:
/ftadv:P=myprofile,d=iso8859-15/testfile
The advice string also sets the dataset codeset to "ISO8859-15". This
value overwrites the value specified in the profile.
Note: Advice strings can also be used with directory names in
file transfer GUIs. With advice strings, file transfer can be controlled
in the GUI. For example in the Windows GUI, if you use the directory
/ftadv:P=WIN/____path/to/current/directory/, all file transfers
to and from the directory /path/to/current/directory use the
advice string /ftadv:P=WIN/. This means that all file transfers
use the named profile WIN (see Section File Transfer Profiles). Note also that the HFS root
directory must be given as ____ (four underscores) because of the
naming convention introduced in the product.
File Transfer Profiles
A file transfer profile is a mechanism for pre-configuring different types
of file transfers. Both mainframe clients and server use the same
profile mechanism. There are two types of profiles: named profiles and
filename-matched profiles.
Named profiles can be used with the file transfer advice string parameter
P. A named profile provides the default values for different
advice string parameters. Those default values can be overwritten with
advice string parameters.
The filename-matched profiles can be used for configuring advice string
default values when transferring files whose names match a certain
regular expression. Again, those default values can be overwritten with
advice string parameters.
File transfer profiles for SSH Tectia Server (M) can be set in the files
/etc/ssh2/ssh_ftadv_config (globally for all users) and
$HOME/.ssh2/ssh_ftadv_config (for the specific user).
The syntax of the profile is:
%NAME|REGEXP ADVSTRING
Each profile starts with a profile name or an egrep style of
regular expression followed by one or more white spaces and the advice
string. The profile name MUST start with the percent (%)
character. The regular expression MUST NOT start with the %
character. If the % character needs to be the first character in
the regular expression, it must be escaped with a backslash (\).
Note that in order to get the regular expression escape character,
backslash, into a regular expression, it must be escaped by using
\\.
The profile name and regular expression MUST start from the beginning of
the line. There MUST NOT be any white space in front of the name or
regular expression.
An advice string is a comma-separated list of name-value pairs of type
name=value. There may be white spaces in the advice string. It
can span over multiple lines. There MUST be at least one white space
character in the beginning of a spanned advice string line. An advice
string MUST NOT contain the file transfer advisor marker /ftadv:/
or a filename.
Profiles with %NAME can be used with the advice string
P=NAME. Profiles with REGEXP are used only for matching
filenames. The first REGEXP that matches a filename is used.
Comments can be added to the file with a hash (#) character.
Everything on the line after # is ignored.
Example File Transfer Profiles
The following profile converts text files from Unix to MVS. ASCII is
converted to EBCDIC. This profile is only used if the advice string
contains the parameter P=UNIX.
%UNIX X=text,
F=line,
C=iso8859-1,
D=ibm-1047
The next profile converts text files from Windows to MVS. ASCII is
converted to EBCDIC. The line delimiter is converted from the Windows
style to the MVS style. This profile is only used if the advice string
contains the parameter P=UNIX.
%WIN X=text,
F=line,
C=iso8859-1,
D=ibm-1047,
I=dos,
J=mvs
The following profile can be used when transferring text files from
MVS to another MVS. This profile is only used if the advice string
contains the parameter P=ZOS.
%ZOS X=text,
F=line
This profile can be used when transferring text files between the Unix
and MVS environments. All datasets created on the MVS side have fixed
blocked lines of 80 characters per line. Again, to use this profile,
P=FB80 needs to be added to the advice string.
%FB80 X=text,
F=line,
C=iso8859-1,
D=ibm-1047,
O=fb,
R=80
The following profile creates record format datasets with maximum record
length of 1024 bytes. The advice string P=REC is needed.
%REC F=record,
R=1024
The following profile can be used when transferring binary files.
%BIN X=bin,
F=stream
This next profile is used whenever a dataset name contains the string
TXT, txt, TEXT or text in it.
//.*(TXT|txt|TEXT|text).*$
X=text,
F=line,
C=iso8859-1,
D=ibm-1047
The following profile matches files that have the extension .txt,
.TXT, .c, .C, .h, .H,
.log, .LOG, .conf or .CONF. Codeset
conversion from ASCII to EBCDIC is performed.
.*\\.(txt|TXT|c|C|h|H|log|LOG|conf|CONF)$
X=text,
F=line,
C=iso8859-1,
D=ibm-1047
The following profile matches binary files with extensions
.gz, .Z, .tar and .bin.
.*\\.(gz|Z|tar|bin)$
X=bin,
F=stream
This profile matches all files and dataset names. Data is transferred
in text format and codeset conversion from ASCII to EBCDIC is
performed.
.*$ X=text,
F=line,
C=iso8859-1,
D=ibm-1047
File Transfer Environment Variables
If no profile is found, the default values for advice string parameters
are taken from the following environment variables:
SSH_SFT_ADVISOR_TRANSFER_MODE
SSH_SFT_ADVISOR_TRANSFER_FORMAT
SSH_SFT_ADVISOR_FILE_CODESET
SSH_SFT_ADVISOR_TRANSFER_CODESET
SSH_SFT_ADVISOR_TRANSLATE_TABLE
SSH_SFT_ADVISOR_TRANSLATE_DSN_TEMPLATES
SSH_SFT_ADVISOR_FILE_LINE_DELIMITER
SSH_SFT_ADVISOR_TRANSFER_LINE_DELIMITER
SSH_SFT_ADVISOR_FILE_TYPE
SSH_SFT_ADVISOR_FILE_ORG
SSH_SFT_ADVISOR_FILE_RECLEN
SSH_SFT_ADVISOR_FILE_BLOCKLEN
SSH_SFT_ADVISOR_FILE_SIZE
SSH_SFT_ADVISOR_DIRECTORY_SIZE
The environment variables for SSH Tectia Server (M) file transfer can be set
in the files /etc/environment (globally for all users) and
$HOME/.ssh2/environment (for the specific user).
