I spent most of last night working on do-fs1's backup volume. As part of the troubleshooting steps, I performed some updates on the Drobo firmware, which in turn forced me to disconnect and reboot the Exchange server (postal), as well as the SQL server (do-sql).
Unfortunately, the problem ended up being in the Windows 2008 Server Backup VHD volume, which apparently had corruption that I was unable to fix. (I suspect that one of the power outages in the recent past may have caused this). As a result, I had to delete that VHD (which are the backups that we had previously, and start over from scratch). This corrupt volume only houses data which resides on do-fs1. So, Exchange, SQL Backups remain untouched.
I started running a full backup on do-fs1 at around 11:30pm, which is significantly later than the normal scheduled backups, so this process may go into late morning before it finishes. I don't suspect that this will affect user access (too much), perhaps a tiny slowness, but nothing that would generate any calls.
During the time when I was doing the troubleshooting, there may have been a period where other backups were scheduled to run. (perhaps do-sql or exchange). In this case, those backups probably failed to run. please schedule them to run normally tomorrow.
Aside from the above, I think we are now back to normal. I will be monitoring those volumes and the do-fs1 backups, and will attempt some test file restores to make sure the backup sets being saved are actually valid.
For any questions/concerns, please contact me.
Unfortunately, the problem ended up being in the Windows 2008 Server Backup VHD volume, which apparently had corruption that I was unable to fix. (I suspect that one of the power outages in the recent past may have caused this). As a result, I had to delete that VHD (which are the backups that we had previously, and start over from scratch). This corrupt volume only houses data which resides on do-fs1. So, Exchange, SQL Backups remain untouched.
I started running a full backup on do-fs1 at around 11:30pm, which is significantly later than the normal scheduled backups, so this process may go into late morning before it finishes. I don't suspect that this will affect user access (too much), perhaps a tiny slowness, but nothing that would generate any calls.
During the time when I was doing the troubleshooting, there may have been a period where other backups were scheduled to run. (perhaps do-sql or exchange). In this case, those backups probably failed to run. please schedule them to run normally tomorrow.
Aside from the above, I think we are now back to normal. I will be monitoring those volumes and the do-fs1 backups, and will attempt some test file restores to make sure the backup sets being saved are actually valid.
For any questions/concerns, please contact me.
- This troubleshooting was performed on 05/22/2012 at 9:30pm
No comments:
Post a Comment
Please make your comment. (GMK)
Note: Only a member of this blog may post a comment.