Server Backup

We cannot over-emphasize the importance of a backup. If your server was stolen or lost in a fire, how would you recover? The goal of the backup is to be able to restore the entire server in the case of an emergency, or one or more files if they were accidentally erased or modified and an earlier revision is required.

The backup process is initiated automatically each night, typically at 2:00AM. In most cases the entire server hard drive or drives are backed up onto a single tape. In some situations, with large capacity disc drives, certain files are excluded from the nightly backup because they would not fit on even one of the largest capacity tapes. In this case a separate manual backup is provided to back up the files that are not backed up each night.

Because the backup process is not foolproof, Dynamo emails the backup results to up to three system administrators as specified in Dynamo Configuration Maintenance (SMC). This email will include the last 30 server backup attempts. An example of the email is shown below. There is also a complete Server Backup Log available on the SMS menu.

The backup can fail for a variety of reasons. Look for your situation below, and follow the steps in the Corrective Action column in an effort to resolve the problem.

Case	Situation	Corrective Action
A	Email indicates: 'Files on Tape' is zero each day no matter what tape is used, or error mssage: cpio: /dev/st0: Read-only file system	Verify that tape has been inserted into tape drive, and did not eject prior to the start of the backup. Verify that the tape is not 'write-protected'. Shutdown & restart the server Run a manual backup from a PC so you can see any error message.
B	Email shows message: cpio: write error: No medium found	Verify that tape has been inserted into tape drive, and did not eject prior to the start of the backup. Verify that the tape is not 'write-protected'. Shutdown & restart the server
C	The Friday/Saturday/Sunday backup fails, but other days work	Since the Friday tape is written to 3 times as often as the other tapes, it is most likely that the tape can no longer be used and should be replaced. Using the recommended schedule, a Friday tape would need to be replaced approximately every 18 months, where a weekday tape should last 48 months before replacing. The tape used on Friday is a lower capacity than the tape used other days. Refer to chart of tapes available below.
D	The same tape, i.e, A-Wed, always fails, yet other tapes backup without a problem, or Email shows error message: cpio: I/O error on write(): I/O error (error 5)	The tape is worn out and must be replaced. Tapes typically can be used about 100 times before replacement. The tape that fails is a lower capacity than the tapes that work. Refer to chart of tapes below.
E	Email indicates: 'Files on Tape' is less than the 'Files to Backup', or The 'Time to Backup' is much shorter than normal, or error message: cpio: /dev/tty: No such device or address error message: cpio: can't open /dev/tty for writing	Files may not all fit on one tape. Use the Dynamo menu D option to show disc usage and verify the tape being used is large enough. Try a different tape with sufficient capacity Erase any files, created from a PC application, that are stored on the server that are no longer needed. Contact Excellware to erase any archived Dynamo files that are no longer needed. Clean the tape drive Shutdown & restart the server Run a manual backup from a PC so you can see any error message
F	Email indicates: 'Files to Backup' is a few numbers higher than 'Files on Tape' error message: cpio: 0511-025 Cannot get information about xxxx	File xxxx was present when the backup started and the list of files to backup was prepared, but was erased by the time the backup was processing that section of the disc. Although the backup is run when there should be minimal activity on the server, there still may be some activity related to outgoing email, scheduled communication activity, or other tasks scheduled during the backup. If the message occurs each day, then the time of the backup or other scheduled activity could be changed not to overlap. If the message occurs only rarely, then the backup is probably complete, and simply excludes the files that were removed while the backup was in progress.
G	Email indicates: 'Files to Backup' are the same as the 'Files on Tape', but the Status shows 'Bad'. The 'Time to Verify' is much lower than normal	Clean the tape drive Verify that the tape is not 'write-protected' Replace tape Shutdown & restart the server

If you have performed all of these steps, yet you still are unable to get successful backups each day, then although unlikely, perhaps the tape drive hardware has failed. Contact IBM at 1-800-IBM-SERV for service. You will need the server machine type and serial number when you call. This information is on a small sticker on the front of the server.

Manufacturer	Item#	DDS number	Tape length (meters)	Approximate Capacity (Gb)	Works with Drives	Price (as of Jan 2009)
Imation	42818	1	90	2 to 4	DDS-1, DDS-2, DDS-3, DDS-4	$6.00
Imation	43347	2	120	4 to 8	DDS-2, DDS-3, DDS-4	$9.00
Imation	11737	3	125	12 to 24	DDS-3, DDS-4, DAT 72	$8.00
Imation	40963	4	150	20 to 40	DDS-4, DAT 72	$11.00
Imation	17204	DAT 72	170	36 to 72	DAT 72	$22.00
Imation	45382	Cleaning Tape	N/A	30-50 cleanings	all	$15.00

Write enabled vs. Write protected

Each 4mm tape has a small slider as shown below. When the slider is 'open' the tape is write-protected, i.e., you cannot write on the tape. This is often done when restoring files from a tape, to prevent accidentally overwriting the information on the tape. The slider must be closed to enable writing (recording) information on a tape.

Erase files created by PC applications that are stored on server

Excellware recommends that no files of importance are stored on the PC. Important files should be stored on the server to insure they are backed up daily, and can be recovered should the hard drive of the PC fail.

However, these PC application files can take up quite a bit if disc space, which will cause the backup to fail, when the data to be backed up does not fit on a single tape.

Use the Windows Explorer (Right Click Start, and left click on Explore) to explore the shared folders located on the server, or click on Start, then Run, and enter \\server in the Open box.

Erase files no longer needed.

The PRT share contains Dynamo reports that were printed to a file, and the export share contains files exported by Dynamo. These too should be erased when no longer needed. You can also use the Dynamo option to 'Manage reports printed to a file' option on the SMU menu to remove files.

If you have erased all of the files you can, yet the backup still does not fit on one tape, then you will need to use a higher capacity tape drive, or exclude some data from the daily backup.

Manual Backup

The backup program that runs at night shows most error message in the backup email. But some error messages cannot be included in the email. In that situation, it is best to run a Manual Backup from a PC, so that you can observe all error messages.

Normally a backup is run when there is no activity on the server, so that the backup could be restored in case of emergency. In this case, however, the purpose of the manual backup is to observe any error messages. Since the backup will not be used to restore data, you can run the manual backup while other users are logged in. You may encounter a message that indicates a file changed while it was being backed up, but that would be expected when running a manual backup while Dynamo is being used.

A user with Console Mode permissions (as set using Operator Info Maintenance on SMS menu), should enter ! at any Dynamo menu.

Then enter su to request super-user (root) permissions.

Then enter root's password when prompted. Contact Excellware if you forgot root's password.

At the # prompt enter: backup.cpio

This will start a manual backup that is very similar to what runs each night. The manual backup will take just as long as the nightly backup, including the verification time.

Look for any messages that might provide a clue as to the cause of the backup. For example, if you have more data to backup then will fit on a single tape, the display will show: Found end of tape. To continue, type device/file name when ready.

When the backup is completed, enter exit at the # prompt to return to the Dynamo menu.

Shutdown & Restart Server

Have all users log out of Dynamo. Verify that all users are logged out using the Dynamo T menu option. Then logout yourself.
If you have a monitor, keyboard, and mouse connected to the server, then use the mouse to click on the word 'Shutdown' server, and then confirm that you want to shut down and power off the server. If you do not have a monitor, keyboard, and mouse connected to the server, then use the eraser end of a pencil to press and hold the small round white power button on the server until all the server lights go out. This takes about 5 seconds.
Wait one full minute.
Then use the eraser end of a pencil to touch the small round white power button on the server to power it back on. The server will take a couple minutes to complete the reboot process before users can log back on.

Technical Description of the Excellware Server Backup process

At the start of the backup, an empty file named /tmp/backup.date is created. You can look at the date and time of this file on the disk to determine the last time the server backup program was run. This empty file is also copied to the backup tape so that later you can determine when the tape was last used. You can also look at the date and time of this file on a backup tape to know when this particular tape was last used for a backup. See the example below.
A list of all files to be backed up is created and named /tmp/backup.disk.
The tape is rewound to the beginning, and all files from step 2 above are copied to the tape.
The tape is rewound again, and the tape is read from beginning to end, making a list of all files found on the tape. The list is stored in /tmp/backup.tape. This step insures that we can read the tape from beginning to end without problems.
Finally, the tape is rewound again, and an internet email is sent to the designated system administrator(s) reporting the backup status. See example email above. The email contains the last 30 times the server backup program was started. You can view older backup activity using the Display Server Backup Log (SMS).

Viewing and Restoring Tapes

The instructions that follow, assume the backup tape was created using Excellware Server Backup program.

The first step is to get to the linux command line. There are several ways to do this depending on your particular situation.

If you use the FacetWin terminal emulator, you can login using your normal login and password, enter an exclamation point (!) at any Dynamo menu, to get to the $ command prompt.
If you have a monitor connected directly to the server, you can login using your regular user name and password, then select System Tools, then Terminal. The Terminal program may be in a different location depending on your version of linux.
If you do not have a monitor connected directly to the server, nor are using the FacetWin terminal emulator, you can get to the linux command prompt from any PC on the network. Click Start, Run, then enter telnet in the box and click OK. At the 'Microsoft Telnet>' prompt enter: open server, then enter your login id and password. Note that if you make an entry error entering your login id, you need to use the Delete key to backup, and not the Backspace or Left Arrow key.
If your server is not configured for telnet you will need to access the server using the putty program. To run this program you will need to click the Start, Run, then enter putty in the box and click OK. Alternatively you can type C:\program files\putty\putty.exe in the box to run. In the dialog that opens type server in the host name and click Open.

For each of the options listed above, you should now be at a linux command line with a $ prompt:

The next step is to become root, the super-user.

At the $ prompt, enter:

su

then enter root's password. If you don't know root's password contact Excellware.

You should how have a # prompt instead of a $.

When did the system backup last?

Follow the steps outlined above to obtain the # prompt, then enter this command to see when the backup was run last. Normally it should indicate today at 2:00AM. If it does not then please contact Excellware to insure the backup program is still being scheduled to run.

ls -l /tmp/backup.date

What files were included in the most recent backup?

The files scheduled to be backed up can be listed with the command:

less /tmp/backup.disk

If you get an error message saying that 'less command not found', then substitute the 'more' command in each example.

The files that were found on the most recent backup tape can be listed with the command:

less /tmp/backup.tape

The 'less' command will fill the screen and pause. You may enter the following commands:

Enter - displays next line in file
space bar or d - displays next page
u - displays previous page
/ - followed by search argument will search file
n - after a search, n will search again for the next occurrence
p - returns to top of list
any number - advances the number of lines specified
q - quits

In many cases, the following keys also work:
Home - top of list
End - bottom of list
PgUp - next page
PgDn - previous page

When was a particular tape last written to?

Insert the tape to be checked into the tape drive, follow the steps outlined above to obtain the # prompt, then enter this command to see when a tape was last used for backup. Note that this may take a while since it will read the entire tape.

Use the same options for the 'less' command listed above, however since it will take some time to read the tape, the search and end options may take a long time before anything displays.

Restoring files from a backup is a serious matter. Be very careful before touching the Enter key, that you have entered the command exactly. If you restore more files than you planned to, you could destroy other files that have not been backed up. If you type the command incorrectly, you could accidentally write on the tape instead of read from it.

We also recommend changing the tape to "read-only" when restoring, so you don't accidentally write on the tape which could happen if the command is entered incorrectly. You can make a 4mm tape "read-only" by opening a small sliding door on the side of the tape. Be sure to close the sliding door before the next time you want to write on the tape.

To restore files into the same location they were backup up, overwriting the current version (if any), be sure there are no users accessing any of the files to be restored, and enter this command to change to the root directory:

To restore files into another location on the disc, use the change directory command (cd) to change to another directory such as /tmp:

where file1 and file2 are the full names of the file as stored on the tape. Use the

command to display the format of the file name. Typically the file names should exclude the leading / and period. For example, to restore two files and replace the existing copies of the files, enter the commands:

To restore a large number of files, it is often easier to create a list of files to be restored and store them in a file using vi or another text editor. The following command restores all files contained within the namelist file /tmp/filenames:

Note the use of quotes when using wildcards. Quotes are also required when a file name contains spaces (a discouraged practice).

Excellware, Inc.
Server Backup
Tape	Started	at	Status	Files to Backup	Files on Tape	Time to Backup	Time to Verify	Done
C-Jan	Wed 02/01/06	2:00	Good	134,500	134,500	01:12:36	01:08:25	4:25
A-Mon	Tue 01/31/06	2:00	Bad	133,299	0	00:00:01	00:00:00	2:00
B-Fri	Mon 01/03/06	2:00	Good	133,227	133,227	01:08:10	01:00:17	4:10