POLYTROPE

polytrope > support >

What to do about damaged database files

IMPORTANT WARNING

This is a very technical and very complicated subject. The information provided here is necessarily partial, may not be pertinent to the problem that you face. Worse than that, things change and the info here might actually be misleading or wrong. Information from FileMaker Inc may supersede the information here. Read this article skeptically and test what we say here against your own knowledge of your setup, your operating system, etc. Proceed at your own risk!

Is your database file damaged? Is it serious?

Database files do get damaged. Damaged files are - or should be - a very rare problem. But the problem is not non-existent.

Sometimes you will get some sort of indication from FileMaker Pro (or FileMaker Server) that a file may be damaged. Perhaps a file won't open in FileMaker Server. Take a look at the log: you may see that the open command failed because the file is suspected of being damaged. If you aren't using FileMaker Server, you may occasionally get an alert from FileMaker Pro indicating that the file is damaged and suggesting that you use the Recover command to fix it. If you see messages such as this, you have a serious problem.

On very rare occasions, the file may open properly, but behave very erratically. The database may freeze when you select a particular record. Data values in fields may suddenly change bizarrely. This kind of problem is hard to diagnose, because it's possible that the erratic behavior you think you see is in fact normal behavior that you don't understand. For example, occasionally we hear from users who think that records have "disappeared" from the database. That particular complaint is almost always due to a misunderstanding on the user's part. Nevertheless, data damage occurs and can cause weird behavior in the database. And this too can be serious.

More frequently, if the files were closed improperly, next time you open the database, you may see a brief alert saying that the file was not closed properly and that FileMaker Pro is checking the file for consistency. If this message goes away quickly, and the files appear to behave normally afterwards, the odds are that the file is fine. But if you see such alerts frequently, it is a sign that something bad is happening, and that bad thing could cause serious damage eventually.

What causes files to be damaged?

You can't always tell that your files are damaged - and when you do suspect damage, it's not always possible to know or even guess what caused it. There are however a couple of common causes.

Power outages and computer crashes seem to be the most common causes of damage to FileMaker database files. If you know that your computer crashed or lost power, and shortly after that event you suspect or have reason to believe that a database file is damaged, there's a good chance that your suspicion is correct. Database files need to be closed properly. If power goes out suddenly, or if the computer crashes or freezes while the database files are open, the database files may be damaged.

There are however a wide variety of other possible causes of damage to database files. Some of these problems are relatively easy to avoid, others are not. Faulty installation and/or opening the files incorrectly over a network can cause damage. Copying files that are open often causes damage. We have known cases where files were damaged apparently because someone was doing development work (in particular, making changes to the database schema like defining fields or relationships) while the files were actively hosted on a network. Hard disks get damaged for their own large set of reasons. Viruses and malware can do harm that directly or indirectly damages databases. Installing bad memory in your computer, waving a huge magnet near the hard disk, conflicts between other applications on your computer: If you want to lie awake worrying, there are more things to worry about than we can enumerate.

FileMaker developers like to joke (nervously) about demonic possession. But this is exceedingly rare. And if your database files truly are possessed by Satan, then you are probably facing more urgent problems than fixing your database.

Restore files from last good backup

You must be backing up your files. See our article on this subject here.

If you conclude or strongly suspect that one or more database files are damaged, and if you can reasonably attribute the damage to an event that you can date - a power outage or crash - then your best recourse is to restore the most recent backup made prior to the damaging event. For example, say your database files are hosted on a machine running FileMaker Server. You have a schedule defined that causes FileMaker Server to backup your files every day at 2am. You come in to work Monday, find that the server crashed or lost power on Sunday, and you have reason to think the database files are damaged. Best solution in this situation: Go to the backups folder on the server, get the most recent backup made prior to the crash or power outage. Close the damaged files on the server, move them to the Trash or Recycle Bin, and move the most recent backup files into the Databases folder, and open them. The theory here is that the backups are not damaged and that putting them online in place of the damaged files is the safest bet.

Note that in the scenario just described, it seems that no changes were made to the database between the time of the last good backup and the time you discovered that you had a problem. What if that's not the case? If changes were made, you can redo the data entry (make the changes again). You should be backing up often enough that you never lose too much data in the event of disaster. FileMaker Inc suggests that you can restore the backups, then recover the damaged files (see below), find the records that have changed since the backup was made, and update the changed records. This may be a good idea, but it's too complicated to go into here.

What if you don't have a recent backup? Then you don't have a lot of truly reliable options. When you stop kicking yourself, your best bet may be to recover the damage file, export the data to a safe format such as merge format, close FileMaker Pro, then import the data into a new copy of the database file that you know to be good.

When to use FileMaker's "Recover" command

Sometimes a FileMaker Pro file can get damaged so badly that FileMaker cannot open it in the normal way. When this happens, FileMaker will advise you that the file is damaged and will recommend that you use the "Recover" command to extract the data from the file. You should not use the Recover command unless FileMaker advises you to do so. Do not recover files as a matter of routine maintenance.

If are not warned that the file needs to be recovered, but you suspect the file is damaged - for example, it won't open in FileMaker Server - and if you don't have a recent backup to restore, you can try opening the database as a sole user in FileMaker Pro and saving a compressed copy of all of the files in the solution. This writes a fresh copy of the files to disk and if the file wasn't badly damaged in the first place - and again, if you have no better alternative - then you may be able to put the fresh copies back into service.

Using the Recover command

If FileMaker tells you that a file is damaged and needs to be recovered, do the following. In these instructions we use "Myfile.fp7" as our example file name. Be sure to substitute the actual name of the damaged file you are dealing with!

1. Close all of the solution's files and quit FileMaker Pro.

You may have to force FileMaker to quit, and if that is the case, it may be necessary or advisable for you to restart your computer before proceeding. NOTE: As you are closing CM Assistant, you may get alerts from FileMaker telling you that it can't find the damaged file and telling you to please locate it. Ignore these requests! Cancel out of all these dialogs.

2. If the solution is shared on a network, make sure nobody else is using it. If the solution is being shared using FileMaker Pro Server, use the FileMaker Pro Server admin console to take the database files offline.

3. Open FileMaker Pro on a machine that will give you direct access to the damaged file.

If the solution files are normally stored on a special server, you may need to copy the damaged file to another computer in order to run the recovery procedure.

4. When FileMaker is open, pull down in the File menu to "Recover." Using the open-file dialog that appears, find the damaged file. Our example damaged file is named Myfile.fp7.

 
FileMaker will now perform the recovery. This may take a minute, a few minutes, or quite a while, depending on the size of the file and the speed of the computer. During the process you will see progress dialogs on screen telling you how the procedure is going. It is a good idea not to try to use your computer during this time. When FileMaker is finished, it will create a new file named "Myfile Recovered.fp7." Now it's your turn again.
 

5. Quit FileMaker Pro.

6. Locate the old (damaged) file. Change its name from "Myfile.fp7" to "Myfile Damaged.fp7" or something similarly useful. DO NOT open this file again.

7. Change the name of the newer file from "Myfile Recovered.fp7" to "Myfile.fp7".

IMPORTANT: The goal here is to replace the old file with the new one and give the new file EXACTLY the same name as the old one. This is the name that your database solution expects the file to have. It is absolutely imperative that the new file end up with precisely the right name: no additional spaces, etc. Failure to rename the recovered file properly could cause your copy of CM Assistant to become unusable.

OK, you've recovered the file. Now what?

Recovered does not equal "fixed": the next step

Our clients have put recovered files immediately back into use and have not reported serious problems. However, FileMaker, Inc.'s official position is that the recovery procedure is designed simply to get the file back into good enough condition so that you can open the file and extract your data from it. At that point, you should move the data in the file into a fresh copy of the recovered file. This official position is based on good knowledge and experience and should be seriously considered. You should not ignore FileMaker Inc's advice here unless you really have no choice.

Users of Polytrope's licensed database solutions (CMAssistant, Goodbooks, etc.) may be able to contact Polytrope Support for help here. We can get your data and move it into a clean copy of the database files. There will be a charge for this service but it will be worth paying.

Final thoughts

It is or should be very uncommon for a file to be so badly damaged that it requires the Recovery command. At Polytrope, we have not had to use the Recover command ourselves at all in the last year or more on a single file, and we handle a lot of files.

There are a couple of things you can do to reduce the risk of serious damage to your files. Keep your hard disks in good shape, using standard hard disk maintenance and repair utilities. Use virus-protection software to keep your hard disk free of destructive viruses. If power outages are a problem, get an uninterruptable power supply (UPS) for your computer. Make sure your computers have plenty of good-quality memory and hard disk storage space. While you are using your database, keep the number of other programs that you have open to a minimu. Perhaps most effective of all, if you are sharing CMAssistant or Goodbooks with other users and if it's within your budget to do so, run the database under FileMaker Pro Server on a dedicated machine. This way, even if your computer crashes very hard, the risk of serious damage to the FileMaker files (housed on the server) is very small.

However, you need to be aware that there is no way to reduce the risk of serious file damage to zero. Even worse, a file can on very rare occasions be damaged so badly that it is beyond the reach of the powerful FileMaker Recover procedure. In other words, you have lost your data. It is imperative that you make regular dated backups of your files to removable media such as tape or CD. At a minimum, backup your files every single week.

Page last modified:  February 19, 2009 11:58
Copyright (c) 2009, Polytrope LLC, Dallas, Texas.