| Alternate Procedure: Use RSYNC to transfer/backup data.
The benefits of using RSYNC over the FTP scripts are:
|
The RI 7100A test system is capable of generating large amounts of data, very quickly. In a typical production environment, it is necessary to move this data quickly and easily from the tester to a central server for processing. This note describes two ways to accomplish this goal.
Data can be either "pushed" from the RI tester to the central server, or the data can be "pulled" by the server from the tester. The rest of this note will refer to these methods as "push" and "pull". The only difference is which computer runs the program that moves the data. In the case of the tester "pushing" the data, the script runs on the tester, under the OS/2 operating system. In the "pull" case, the program runs on the server, under whatever operating system the server uses (often Linux).
Sample scripts are provided for both "push" and "pull" scenarios. In the "push" case the script is in Rexx, a scripting language that comes with the OS/2 operating system. Rexx is a very "basic-like" programming environment well suited to scripting tasks. In the "pull" case the script is in Python, an object-oriented scripting language available across many platforms (including OS/2, Linux, and Windows). In either case, the provided script is a starting point for your own script. It shows one possible way to accomplish the task, not the only way or even the "best" way for your circumstances.
ATTENTION: Always transfer .ZIP files with FTP in BINARY mode. ASCII mode will corrupt .zip files because of the line feed conversion (nl -> nl+cr). To put the standard FTP client into binary mode enter the command binary before starting the file transfer.
Data Transfer Basics
Reliable data transfer is accomplished with the FTP software protocol. This robust file transfer protocol has served as the workhorse of the Internet for many years and is extremely reliable. There are always problems that can occur during transfer, however, so we recommend a "three step" transfer process:
Step 1: The computers use FTP to transfer any new files from the Roos Instruments tester to the server.
Step 2: The server processes the new data files and marks them as "finished" after processing. This may be done by renaming the files in a particular way.
Step 3: The computers compare "finished" files on the server to the files on the tester and remove matching files from both computers. Then loop back to Step 1 after some waiting period.
By "closing the loop" in this manner you insure that valuable test data is never lost in the transfer or processing steps. If a problem occurs on the server, simply wipe out its directory of testdata and restart the transfer process. It will re-copy all of the data from the testers and reprocess the files.
Sample "Pull" Script for Server
Refer to the attachment xferftp.py below. This Python program can be installed on any computer that supports the Python language. The home page of Python is http://www.python.org. It is an open-source programming language that has been ported to a very wide variety of computer platforms.
At the beginning of this script you will see a "user customization" area:
| # vvvv USER SETTINGS vvvv # enableDelete = True #safety to control deletion of files after processing sleeptime = 600 #time in seconds between loops loop = True #set to 'True' to loop forever, 'False' to run once #localpath below is the destination of all csv files transferred localpath = '/var/testdata/csv' #remotepath below is an array of: # (IP address,logon name,logon password,remote dir,system name) remotepaths = [ ('192.168.1.1','username','userpass','d:/riapps/testdata/csv','Roos01') ('192.168.1.2','username','userpass','d:/riapps/testdata/csv','Roos02'), ('192.168.1.3','username','userpass','d:/riapps/testdata/csv','Roos03')] # ^^^^ USER SETTINGS ^^^^ # |
If you examine the script you will understand the meaning of these variables. The first three, "enableDelete", "sleepTime", and "loop" affect the overall behavior of the script. The "localpath" is the directory which will receive the files and the "remotepath" is an array that contains the logon information and directory path for each tester in the network.
This script expects the files to be renamed by the server after they are processed. A file named "1234.CSV" will be considered to be finished if its name is changed to "1234.X". Finished files are then deleted from both the server and the tester.
Sample "Push" Script for Tester
The sample for pushing data from the tester to the server is simpler. It does NOT do the recommended three-step process that insures data integrity, but it could be modified to provide this extra protection. Refer to the attachment RiFtpPut.cmd below. This version is written in the Rexx programming language, which is a built-in scripting language in OS/2. Rexx was created by IBM, which still maintains the best web site for Rexx information at: http://rexx.hursley.ibm.com/rexx/
Near the top of the file you will find a user customization area:
| /* ** Initialize constants used in program */ Host = 'bigserver' /* Name of host computer */ Logon = 'username' /* Logon to use */ Password = 'userpass' /* Password */ HostDir = '/var/testdata/csv' /* Remote host directory */ SecondsToWait = 600 /* Seconds between checks for new data */ GraceSeconds = 1 /* Seconds after data file found */ errLogFile = '\riapps\testdata\RiFtpPut.Err' /* Daemon logs to this file */ |
Again, examine the program to understand the meaning of the variables in the user customization area. The host name, logon, and password should gain FTP access to your server (where the files will be sent). The hostdir is the directory on the host system where the files will be placed. SecondsToWait provides a delay between runs of the FTP program and GraceSeconds pauses a short time between the time the file is discovered and the time it is sent, to give the RI System Software extra safety margin to finish writing the file before it is sent. Messages from this script are copied to the file errLogFile so they can be reviewed later in the event of a problem.
Conclusion
Using these programs as an example, you should be able to create a script that will automate the transfer of data from your RI 7100A tester to a server. Please contact [email protected] if you would like us to provide customization for your particular installation.