by Paul Beach
Two parameters in the firebird.conf have been available since the introduction of Firebird version 1.5
MAX UNFLUSHED WRITE TIME
- Purpose:
- Use the MaxUnflushedWriteTime parameter to specify how often pages are written to disk when the ForcedWrites parameter is set to Off
- Syntax:
- MaxUnflushedWriteTime = integer
- Semantics:
This parameter determines the maximum length of time that pages withheld for asynchronous writing are flushed to disk when Forced Writes are disabled (asynchronous writing is enabled). Its value is an integer which sets the interval, in seconds, between the last flush to disk and the setting of a flag to perform a flush next time a transaction commits.
The default for this parameter is 5 seconds in Windows installations and -1 (disabled) in installations for all other platforms.
MAX UNFLUSHED WRITES
- Purpose:
- Use the MaxUnflushedWrites parameter to specify how often pages are written to disk when the ForcedWrites parameter is set to Off.
- Syntax:
- MaxUnflushedWrites = integer
- Semantics:
This parameter was introduced in FB1.5 to handle a bug in the Windows (9x.ME) operating systems, whereby asynchronous writes were never flushed to disk except when the Firebird server underwent a controlled shutdown. Hence, on 24/7 systems, asynchronous writes were never flushed at all.
This parameter determines how frequently the withheld pages are flushed to disk when Forced Writes are disabled (i.e. asynchronous writing is enabled). Its value is an integer which sets the number of flushes caused commits or rollbacks to be withheld before a full flush is flagged to be done the next time a transaction commits.
Default is 100 in Windows installations and -1 (disabled) in installations for all other platforms. This tells the Firebird server to flush pages to disk when CCH_flush calls have been made by 100 commits or rollbacks.
If the end of the MaxUnflushedWriteTime cycle is reached before the count of withheld pages reaches the MaxUnflushedWrites count, the flush is flagged immediately and the count of withheld pages is reset to zero.
Turning off forced writes on a database using gfix (gfix –write) and setting the following in the firebird.conf file:
MaxUnflushedWrites = -1 MaxUnflushedWriteTime = 5
Will have the following effect (assuming the server has been restarted).
What you have now is asynchronous writes of changed pages in the database cache to disk approximately every 5 seconds (assuming that transactions are committing on a regular basis). This is the default setting when forced writes are turned off. In other words with forced writes off the Firebird engine will collect and write pages (depending on cache size) to the Firebird cache as usual, these writes will also go to the filesystem cache with a non-guaranteed possibility to be flushed to disk by the Operating System in the background (pre Firebird 1.5). However, the setting of MaxUnflushedWriteTime to 5 ensures that the filesystem cache will be synchronously flushed to disk with the next commit (if it happens after 5 seconds from the prior one).
One of the problems with Windows and Disk Controllers is you cannot predict when Windows or the controller will actually flush changes to disk, you can only guess (and hope) it will not happen so late, that you have accrued too many changes in the filesystem cache that will render the database on disk unusable should anything happen.
Taking an approach like this can give you a compromise between the old fast-but-never-flushed (forced writes off, pre Firebird 1.5) and the low-but-reliable forced writes on careful write mechanisms. Forced uses careful writes* every commit, whilst this approach gives careful writes on commit after waiting five seconds. If your system has a UPS (Uninterruptable Power Supply) you get more performance for little risk.
Note
There is still a probability of corruption, but there is always a small possibility with forced writes on. Forced writes on only guarantee consistency between the database cache and what’s sitting on the disk.
These settings were introduced to minimise the effect of setting forced writes off, having all the writes flow into the disk cache, and then relying solely on the operating system and disk controller to flush the changes to disk when it felt like it. The dire warnings that accompany turning forced writes off reflect the state of the world before these parameters were introduced.
So if you must have a guarantee that changes are being flushed to disk on a regular basis (a timed guarantee) and forced writes are off then the configuration MaxUnflushed* settings can be your best friend.
For the record forced writes are turned on by default since the release of Firebird 1.0 on Windows, and on Linux the forced writes on policy only came into effect with Firebird 2.1 Beta 2 due to a bug with the fcntl() system call and the flag O_SYNC.
Another point to consider, once Windows becomes a domain controller, it forces the disk write-through cache to be turned off and you cannot turn it on manually. As such it makes any disk writes noticeably slower. If you use synchronous writes in your Firebird database (forced writes on), it will have a significant effect on any Firebird database application. With asynchronous writes (forced writes off) the situation should be much better, However generally speaking, using a Domain Controller for a database server is not a good idea if you care about both performance and reliability.
However if your Windows Server is attached to a serious disc subsystem, e.g. a hardware RAID controller or some sort of Storage Area Network or Networked Attached Storage, then disabling the Hard Disk cache would be the correct behaviour and there would be no performance penalty will because the disc subsystem would have its own battery or flash backed cache.
Microsoft have an interesting article on file caching which can be found at this location.
Forced writes on Windows uses the FILE_FLAG_WRITE_THROUGH value.
With forced writes off and the MaxUnflushedWriteTime set we call the FlushFileBuffers function when the time setting is hit.
The FILE_FLAG_NO_BUFFERING is set using the following firebird.conf parameter
FILE SYSTEM CACHE THRESHOLD
The threshold value that determines whether Firebird will use the file system cache or not. File system caching is used if the database cache size in pages (configured explicitly in database header or via DefaultDbCachePages setting) is less than FileSystemCacheThreshold value.
To use a file system cache always set FileSystemCacheThreshold to a large value. To bypass file system cache for all databases set FileSystemCacheThreshold to zero.
Type: integer, measured in database pages
Per-database configurable.
FileSystemCacheThreshold = 64K
In other words if FileSystemCacheThreshold is less than the database cache size in pages (including the case where FileSystemCacheThreshold is set to 0) then the FILE_FLAG_NO_BUFFERING will be used.
If you wanted Firebird to never explicitly flush its cache and rely solely on the operating system in the background you can set the following:
Forced writes = OFF (at the database level)
in the firebird.conf:
MaxUnflushedWrites = -1, MaxUnflushedWriteTime = -1
However this is a somewhat more risky strategy if you are trying to maintain some sort of database consistency on disk.
Check out the function PIO_force_write in pag.cpp for more details.
For more information on careful writes Firebird for the Database Expert: Episode 3 - On disk consistency.