Journal
The TSM journal can be your friend or enemy. Which one depends on when you use it and how. IBM have released an FAQ around the journal and below I point out a couple of important areas. It will take you no longer than 60 seconds to improve/refresh you're understanding of the journal.
60 second intro
- Journal Backup is appropriate for backing up files systems with small or moderate amounts of change activity between backup cycles.
- A traditional incremental backup of the file system must be completed while the
journal for the file system is active. - This backup must result in the Last Backup Completion Date being updated on
the TSM server. - Once the full incremental backup has been completed, the change journal is marked
as valid for the TSM client node and server the backup was performed with.
Below is the full document from IBM
TSM Journal Based Backup Frequently Asked Questions
What is Journal Based Backup and how does it differ from traditional
incremental backup ?
Journal Based Backup is an alternate method of backup which utilizes a change journal
maintained by the TSM Journal Daemon process.
The primary difference between traditional incremental backup and journal based backup
is the method in which candidates for backup/expiration are obtained.
Traditional incremental backup obtains the list of backup/expiration candidates by
building comprehensive lists of local objects and lists of active server objects for the file system being backed up.
The local lists are obtained by scanning the entire local file system. The server list is
obtained by querying the entire server inventory for all active objects.
The two lists are compared, and candidates are selected according to the following
criteria:
An object is selected as a backup candidate if it exists in the local list but doesn't
exist in the server list, or if the object exists in both lists but differs according to
TSM incremental criteria (attribute changes, date/size changes, etc.).
An object is selected as an expiration candidate if it exists in the server list but doesn't
exist in the local list.
Journal based backup obtains the candidates list of objects to backup/expire by querying
the TSM Journal Service for the contents of the change journal of the file system being
backed up.
Change journal entries are cleared (marked as free) after they have been processed by the backup client
and committed on the TSM server.
Journal Backup is activated by configuring the journal daemon to monitor specified file systems
for change activity and is enabled by successfully completing a full incremental backup.
What platforms and TSM client versions is Journal Backup available for?
Journal Based Backup is supported on all supported TSM Windows platforms and on AIX as
of TSM client version 5.3.4.
Journal Based Backup was introduced in the TSM version 4.22 client and in the TSM 5.3.4 AIX client.
A major revision in client version 5.30 added an improved file system monitor, a more efficient journal database, and multiple journal backup session support.
As of this writing, the most up to date version of this function is TSM client version 5.40.
Recommend Usage and Limitations
Journal Backup is appropriate for backing up files systems with small or moderate amounts of change activity between backup cycles.
Very large amounts file changes between backup cycles will result in very large change journals,
very large journals pose memory and performance problems which may negate the usefulness
of journal backup.
Creating, deleting, or renaming/moving very large directory trees may also negate the benefit of using
journal backup instead of normal incremental backup
Journal backup is not intended to be a complete replacement for traditional incremental backup.
The goal of journal backup is to track all file system changes in the change journal and
to make every attempt to keep the journal synchronized with what has been backed up.
Since individual server attributes aren’t available during a journal backup, certain policy
settings such copy frequency and copy mode may not be enforced.
Other platform specific idiosyncrasies may prevent objects from being processed properly,
and other software which changes the default behavior of the file system may prevent
file system changes from being detected.
Traditional incremental backup compares the entire server inventory of files against the entire
local file system and will therefore always be the most comprehensive backup method.
Because of this and the stated limitation of journal backup, it is recommended that periodic
full incremental backups be done along with more frequent journal backups.
What are change journals and how are they created/maintained ?
A change journal is a database of file system change information.
The TSM Journal daemon creates and maintains a change journal for each
file system being journaled.
Journal file systems are specified in the TSM Journal Service configuration file
tsmjbbd.ini (see Configuration section for examples).
The TSM Journal daemon monitors each journal file system for change activity and
creates/updates entries in the corresponding change journal which correspond to
the changes.
Each change journal entry reflects the most recent change activity for a particular
object so there can only be one journal entry per object.
A change journal for a file system is activated by adding the file system to the
list of journal file systems in the Journal daemon configuration file tsmjbbd.ini
(see Configuration section for examples).
A change journal must be initialized and validated for a particular TSM client node and server before backups of the file system will be journal based.
How are change journals initialized and validated ?
A change journal must be activated as previously described before it can
be initialized and ready for use.
A traditional incremental backup of the file system must be completed while the
journal for the file system is active.
This backup must result in the Last Backup Completion Date being updated on
the TSM server.
Once the full incremental backup has been completed, the change journal is marked
as valid for the TSM client node and server the backup was performed with.
Subsequent backups by the same client node to the same TSM server will be journal
based provided the journal remains active and valid.
Backing up by a different node and/or to a different server will not be journal based, but
the change journal will remain valid to the original node and server until the change journal
is deleted.
The backup will be use the journal when the following conditions are true:
- backup must be incremental of either an entire file system or a directory
- the journal daemon process in running
- the journal daemon is configured to monitor the file system being backed up
- the change journal is in the valid state
- the backup is using the same TSM node and server the journal is valid for
- all filespace integrity tests pass
- a full incremental backup has successfully completed when the above two conditions are true
- this initial backup
- must specify a full file system (must update "Last Backup Completed " date on TSM server)
- may use different memory efficient option settings including Disk Cache
- may be incremental by date
How is Journal Based Backup invoked and what indication is there
that a backup is journal based ?
A journal based backup is started by issuing a normal backup command with a TSM
backup client (command line, GUI, scheduler, or Web client).
A backup will automatically be journal based if an active and valid journal exists for the
file system being backed up.
The TSM Journal daemon must be started, and the journal must be active and valid for
the TSM node/server specified by the backup client.
If any of these conditions isn't true the client will revert to a traditional backup.
When a backup is journal based, the backup client will display the following
messages when the journal backup begins:
"Querying Journal for '\\pete\f$'"
Processing 22 Journal entries for '\\pete\f$
Note that optimization processing by the journal daemon filters journal entries sent to the client, and the number of entries processed by the client may be much less than the actual number of entries in the journal.
Also note that journal backup uses special logic to process rename directory journal entries.
When a directory is renamed, the files system reports two changes: one for the old directory which was deleted as the result of the rename, and one for the directory which was created as the result of the rename.
Since changes aren’t reported for underlying files and subdirectories, journal backup processing must force full recursive incremental processing on rename journal entries in order to ensure that all underlying files and subdirectories are backed up and expired correctly.
The result is that the journal backup client output generated while processing rename entries appears as a “backup within a backup†as “Successful incremental completed†message will be displayed for each directory which is processed.
How is Journal Based Backup Installed/Enabled ?
On Windows, Journal Based Backup is enabled by installing the TSM Journal Service.
This may be done either by launching the TSM Journal Setup Wizard from
the TSM GUI Client, or by using the TSM dsmcutil command line utility.
Once the service has been installed, any file systems to journal are added to the
list of journal file systems in the journal service configuration file tsmjbbd.ini.
Please see the Configuration Sample section for examples.
The Journal Setup Wizard is capable of installing and configuring the journal service and
making basic journal configuration changes, but many more advanced settings require
editing the journal configuration file directly.
On AIX, the Tivoli Storage Manager journal daemon can be configured by editing the journal daemon configuration sample file, tsmjbbd.ini.smp, and saving it as tsmjbbd.ini. Both the configuration sample file and the saved file should be in the default install directory. The journal configuration file, (tsmjbbd.ini), needs as a minimum a list of the file systems to monitor. These two lines are sufficient:
[JournaledFilesystemSettings]
JournaledFileSystems=/home
After the configuration file is created, start the journal daemon using the script file:/usr/tivoli/tsm/client/ba/bin/rc.tsmjbb. The journal will write initialization information to the errorlog. When you are satisfied that the journal is working correctly, you should run the script file, /usr/tivoli/tsm/client/ba/bin/jbbinittab. Running the script file will make entries in /etc/inittab, so that the journal will begin running when you restart your system.
On both AIX and Windows systems once the journal service has been installed and started, backups will be journal based after a journal for a particular file system has been initialized and validated as described in the section above.
Backups will remain journal based for a particular file system as long as the journal service
is running, the journal remains valid, and the backup is being performed by the same client
node to the same TSM server for which the journal is valid.
Journal Based Backup is always the default backup method used, meaning that if the above
conditions are met the backup will always be journal based.
If Journal Based Backup isn't possible the client will always revert to a normal non-journal based
backup.
By default, journals are always invalidated and deleted when the journal service terminates,
which means that a full incremental backup must be done when the service restarts in order
to re-validate the journal.
This behavior may be overridden with the PreserveDbOnExit setting (described in the Configuration Section).
Why isn't a backup journal based when the journal daemon is running ?
There are a number of reasons why a backup may not be journal based.
An active and valid journal must exist for the file system being backed up for
the backup to be journal based.
A journal is active for a particular file system when the journal service has been
configured to journal the file system (see Configuration setting).
A journal is valid for a particular TSM node and server when a full incremental backup
has been completed when the journal is active.
Note that the full incremental backup command must update the Last Backup Completed
date on the TSM server. If it doesn't the journal won't be valid.
For example, the backup command dsmc incr c: updates the Last Backup Completed date
on the server, but the backup command dsmc incr c:\*.* -subdir=y does not.
The Query FileSpace command may be issued to determine if the Last Backup Completed
date has been set on the server.
A journal is only valid for a particular node and server, so If a subsequent backup is performed by
a different node and/or to a different server the backup won't be journal based, but the journal
will remain valid for the original node/server.
By default, a journal is always deleted (and therefore invalidated) whenever a file system is brought offline or when the journal service is terminated unless the PreserveDbOnExit configuration setting is specified. This setting is described in more detail under the Configuration section.
Certain server changes may also cause a journal to be invalidated.
The file space on the server must conform to the following rules for a journal to be considered valid:
1. The files space must exist on the server.
2. The last backup start and completion dates for the file space must be set on the server
3. The file space must not have been deleted since the last backup completed.
4. The node policy date must not have been updated since the last backup completed.
Note: The deletion date is set by the server whenever some non-client action (server restore, rollback, audit, etc.) modifies the server version of data in a file space such that it can no longer be guaranteed to represent what was on the client at the last incremental backup.
If any of the four listed conditions is not true, the file space is said to "lack integrity" and requires a journal reset and successful full incremental backup to complete before the journal will be available for backup again..
Other errors which occur in the journal daemon may cause the journal to be reset and require a successful incremental backup to complete before the journal will be available for backup again.
Some common journal daemon errors include disk errors (out of space, inaccessible journal, etc.) or other platform specific errors.
On Windows, very high volumes of file system activity may also cause a journal to be reset and invalided due to buffer overflow errors.
This situation is discussed in detail in the Notification Buffer Overflow and Maximum Journal Size Exceeded topics under the Common Problems section.
How is Journal Based Backup configured to run in a cluster environment ?
Cluster exploitation is achieved by configuring a journal service on each node in the cluster to
journal both local and shared resources.
Shared resources are defined with the DeferFsMonStart setting enabled so that they may move
to different nodes in the cluster when a failover or resource move occurs.
The location of the journal directory for shared resource journals must be accessible by all nodes
in the cluster which may own the shared resource, so It may be necessary to use the JournalDir
setting to define different journal directory locations for shared and local resources.
The PreserveDbOnExit setting may be used to to allow journals for shared resources to remain
valid when a resource moves to a different node in the cluster.
On AIX, two sample perl scripts are provided with the journal: jbbadd.pl and jbbdelete.pl. The invocation of these scripts can be added to StartClusterTsmClient.sh and StopTsmClient.sh. The arguments for these scripts are just the file system names of the shared resources.
Cluster configuration settings are described in more detail in the Configuration section.
Journal Service Configuration Settings
Journal service configurations settings are specified in the journal service stanza based configuration file tsmjbbd.ini.
Note that journal service configuration processing is totally separate from backup client options processing.
JournalSettings stanza setting
General journal service process settings.
Tracefile - jbb process tracefile
Traceflags - space delimited set of traceflags
TraceSegMax - maximum size in meg of tracefile segments
TestFlags - space delimited set of testflags
NlsRepos - nls message repository
Errorlog - jbb process errorlog, default is jbberror.log
JournalPipe - session manager pipe name client initially connects to,
used for running multiple instances of the journal service on Windows,
corresponds to client option of the same name
Under most circumstances these settings shouldn't need to be
changed.
Journaldir
The directory where journal database files are stored and written.
On Windows, the default is the Journal Service install directory.
On AIX, the default is a directory named .tSm_JoUrNaL
(used within each file system being journaled).
By default this setting applies to all journaled file systems
but may be overridden via an override stanza for each journal file system.
If the default value is a fully qualified path (for example c:\tsmjournal or /tsmjournal)
all journal db files will be written to the specified directory.
If the default value does not specify a drive letter (for example \tsmjournal)
or start with a directory delimiter (for example ./tmsjournal), the journal db files
for each journal file system will be written to the specified directory on each journal file system.
Not specifying a drive letter or fully qualified name guarantees that a journal will
always be accessible from any node in a cluster if the resource being journal is moved.
JournalExcludeList stanza
Specifies objects not to insert into an active journal.
Filesystem changes to objects which match exclude criteria are not
recorded in the journal.
Patterns may contain simple wildcard characters and environment variable
specifications as follows:
%EnvVar% - expand environment variable
Note that environment variable expansions are done prior to
pattern matching.
Pattern matching characters:
* - match zero or more characters
% - match one character
Note that backup client exclude processing and journal daemon exclude processing are totally separate.
JournaledFileSystemSettings stanza
Default settings for all file systems being journaled.
These settings may be overridden by override stanzas for individual journaled file systems.
JournaledFileSystems
The space delimited list of filesystems to monitor and journal.
Only local, fixed drives are supported, network and removable
filesystems are not.
AIX virtual mount points or Windows 2000 mount points may also be specified (i.e.c:\mountpntdir).
Journals may be brought online or offline while the journal service is running
by modifying this setting.
JournalDBSize
Maximum journal database size (in bytes) for a journaled filesystem.
A size of 0 indicates that the journal database size will be restricted only by
the capacity of the drive where journal db directory resides or by the supported
journal db size of 2 gigabytes.
NotifyFilter (Windows only)
Specifies what types of filesystem activity to monitor for a journaled filesystem.
Multiple activities may be monitored by adding values together.
The default value of this setting may be overridden for individual file systems
via an override stanza.
Specifying a less comprehensive filter value may reduce the size of journals and
improve performance.
The qfilter tool is useful for calculating different filter values, and the filemon file system
profiling utility may be useful in evaluating the amount of change activity generated by
different filter values.
These tools are described in the Useful Tools and Utilities section.
The default value is 0x117 (File and Dir name changes,
last write changes, attribute changes, size changes, and
security changes).
Notification Type Filter Value
File name changes, including create, 0x00000001
delete and rename
Attribute changes 0x00000004
Size changes, notification is deferred 0x00000008
until cache is flushed
Last written time changes, notification 0x00000010
is deferred until cache is flushed
Last access time changes 0x00000020
Creation time changes 0x00000040
Security (acl) changes 0x00000100
DirNotifyBufferSize
Change notification buffer for directory name changes.
By definition a directory name changes occurs when a directory
is created or deleted.
In general this buffer size can be smaller than the buffer size for
other change notifications.
NotifyBufferSize
Change notification buffer size for non-directory name changes on
a journaled filesystem.
A large amount of change activity on a journaled filesystem may
require this to be increased.
On Windows too small a value may caused notification buffer overflows to occur which
will result in journals being invalidated.
On AIX, filesystem activity may be momentarily blocked.
Notification Buffer Overflows are described in more detail in the
Common Problems section.
On Windows, the default value of this setting 0x100000 (1 Meg), and in general
the value should be between 1 and 10 megabytes.
On AIX, the default is 128k.
Notification Buffer Overflows are described in more detail in the
Common Problems section.
The default value specified in this setting may be overridden for individual
journaled file systems by an override stanza.
Note that a notification buffer is allocated for each file system being
journaled, so increasing the value of this setting will increase the
memory utilization of the journal service process.
PreserverDBOnExit
This setting allows a journal to remain valid when it is brought offline.
Journals go offline when the journal service shuts down or when the file system
is removed from the list of journal file systems in the journal configuration file
while the journal service is running.
A value of 1 indicates that the journal will not be deleted when the journal file system
goes offline and will be valid when the journal file system comes back online.
Note that a journal must be valid for backup to be journal based, if it isn't valid
a full incremental backup must be done to revalidate the journal.
This value should be used with caution as any file system change
activity which occurs while the journaled file system is offline
will not be reflected in the journal database.
The default value of this setting is 0 (delete journal db on exit).
This value is useful for preventing journal from being invalidated during
system reboots or cluster failovers.
The default value of this setting may be overridden for individual file systems
with an override stanza.
DeferFsMonStart
This setting allows the journal service to defer bringing a journal online
when the file system being journaled isn't available.
A value of 1 will defer bringing a journaled file system online until the
specified journaled filesystem is valid/available and the specified journal
directory can be created/accessed.
The journal service will also monitor online journaled filesystems for availability
and will bring the filesystem offline if it becomes unavailable.
Resources are checked at the interval specified by the DeferRetryInterval setting.
This setting is most commonly used in a cluster environment where shared
resources may move to different nodes in the cluster.
The default setting is 0, meaning that unavailable or inaccessible journaled
file systems will not be restarted until the journal service is recycled.
The default value of this setting may be overridden for individual file systems
with an override stanza.
DeferRetryInterval
The interval in seconds the journal service will check journaled filesystems configured
with the DeferFsMonStart setting enabled for availability.
The default value is 5 seconds.
The default value of this setting may be overridden for individual file systems
with an override stanza.
LogFsErrors
A value of 1 indicates that all errors encountered accessing
a journaled filesystem or journal directory should be logged
(both to the jbb errorlog and the Windows eventlog).
A value of zero indicates that logging of errors encountered
while checking deferred file systems and journal directories
will be suppressed.
Usually used in conjunction with the deferFsMonStart setting to
eliminate excessive File System unavailable messages from being
written to the logs when bringing a journaled filesystem online
is deferred.
The default value is 1 (log all errors).
The default value of this setting may be overridden for individual file systems
with an override stanza.
Sample Journal Configuration File
;
; JournalSettings - General Journal Service Settings
;
; Tracefile - jbb process tracefile
; Traceflags - space delimited set of traceflags
; TestFlags - space delimited set of testflags
; NlsRepos - nls message repository, default is dscameng.txt
; Errorlog - jbb process errorlog, default is jbberror.log
;
;
; Journaldir - Directory where journal database files are stored and
; written.
;
; Default is the Journal Service install directory.
;
; By default this setting applies to all journaled
; file systems but may be overridden via an override
; stanza for each journal file system.
;
; If the default value is a fully qualified path
; (for example c:\tsmjournal) all journal db files
; will be written to the specified directory.
;
; If the default value does not specifiy a drive
; letter (for example \tsmjournal) the journal db
; files for each journal file system will be written
; to the specified directory on each journal file system.
;
;----------------------------------------------------------------------------
;
[JournalSettings]
Errorlog=jbberror.log
;
;
;
; JournalExcludeList
;
; Specifies objects which will be filtered from being inserted/process
; to/from the journal database.
;
; Filesystem changes to objects which match exclude criteria are not
; recorded in the journal.
;
; Patterns may contain simple wildcard characters and environment variable
; specifications as follows:
;
; %EnvVar% - expand environment variable
;
; Note that environment variable expansions are done prior to
; pattern matching.
;
; Pattern matching characters:
;
; * - match zero or more characters
; % - match one character
;
;
;
; Note that this list contains system files which should always be excluded.
; Please do not remove any of these entries.
;
[JournalExcludeList]
%SystemRoot%\System32\Config\*
%SystemDrive%\Adsm.Sys\*
%TEMP%\*
%TMP%\*
%USERPROFILE\*\index.dat
*.jdb.dir
*.jdb.pag
*.jdbinc.dir
*.jdbinc.pag
*:\pagefile.sys
*:\*.crmlog
*:\hiberfil.sys
%SystemRoot%\csc\*
%SystemRoot%\System32\DTCLog\MSDTC.LOG
%SystemRoot%\netlogon.chg
%SystemRoot%\schedlgu.txt
%SystemRoot%\registration\*.clb
%WINDIR%\debug\*
%SystemDrive%\Documents and Settings\NetShowServices\ntuser.dat
%SystemDrive%\Documents and Settings\NetShowServices\Local Settings\Application Data\Microsoft\Windows\UsrClass.dat
%USERPROFILE%\ntuser.dat
%USERPROFILE%\Local Settings\Application Data\Microsoft\Windows\UsrClass.dat
;
;
;
;
;
; JournaledFileSystemSettings
;
; JournaledFileSystems
;
; Space delimited list of filesystems to monitor and journal.
;
; Only local, fixed filesystems are suppored, network and removable
; filesystems are not.
;
; Windows 2000 mountpoints may also be specified (i.e.c:\mountpntdir).
;
; There must be at least one entry in this list or the Journal service
; will not start. If the service does not start, the reason may be
; found in the event viewer application log.
;
;
;
; Note the following settings specified under the JournaledFileSystemSettings
; stanza will apply to all journaled filesystems unless overriden by a
; JournaledFileSystems override stanza setting of the same name.
;
; For example, the override stanza name for C:\ would be
; JournaledFileSystemSettings.C:\ .
;
;
; JournalDBSize
;
; Maximum journal database size for a journaled filesystem.
;
; A size of 0 indicates that the journal database size will be
; restricted only be the capacity of the drive where journal db
; directory resides.
;
; NotifyFilter
;
; Specifies what types of filesystem activity to monitor for
; a journaled filesystem.
;
; Multiple activities may be monitored by combining (logical OR)
; values.
;
; The default value is 0x117 (File and Dir name changes,
; last write changes, attribute changes, size changes, and
; security changes).
;
;
; Notification Type Filter Value
;
; File name changes, including create, 0x00000001
; delete and rename
;
;
; Attribute changes 0x00000004
;
; Size changes, notification is deferred 0x00000008
; until cache is flushed
;
; Last written time changes, notification 0x00000010
; is defered until cache is flushed
;
; Last access time changes 0x00000020
;
; Creation time changes 0x00000040
;
; Security changes 0x00000100
;
;
;
; NotifyBufferSize
;
; Change notification buffer size for a journaled filesystem.
;
; Large amount of change activity on a journaled filesystem may
; require that this to be increased. The default is 0x100000 (1 Meg).
;
;
; JournalDir
;
; See description under JournalSettings
;
;
; PreserverDBOnExit
;
; A value of 1 indicates that the journaled file system journal database
; will not be deleted when the journal file system goes offline and
; will be valid when the journal file system comes back online.
;
; This value should be used with caution as any file system change
; activity which occurs while the journaled file system is offline
; will not be reflected in the journal database.
;
; The default setting is 0 (delete journal db on exit).
;
;
; DeferFsMonStart
;
; A value of 1 will defer bringing a journaled file system online until the
; specified journaled filesystem is valid/available and the specified journal
; directory can be created/accessed.
;
; Also will monitor online journaled filesystems for availability
; and will bring the filesystem offline if it becomes unavailable.
;
; Resources are checked at the interval specified by the deferRetryInterval
; setting.
;
; This setting is most commonly used in a cluster environment where shared
; resources may move to different machines in the cluster.
;
; The default setting is 0, meaning that unavailable or inaccessible journaled
; file systems will not be restarted until the journal service is recycled.
;
;
; DeferRetryInterval
;
; The value in seconds journaled filesystems with the deferFsMonStart setting
; enabled will be checked for availability.
;
; The default value is 60 seconds.
;
;
; logFsErrors
;
; A value of 1 indicates that all errors encountered accessing
; a journaled filesystem or journal directory should be logged
; (both to the jbb errorlog and the NT eventlog).
;
; A value of zero indicates that logging of errors encountered
; while checking deferred file systems and journal directories
; will be suppressed.
;
; Usually used in conjunction with the deferFsMonStart setting to
; eliminate excessive File System unavailable messages from being
; written to the logs when bringing a journaled filesystem online
; is deferred.
;
; The default value is 1 (log all errors).
;
;
[JournaledFileSystemSettings]
;
; List of journaled filesystems: c:, d:, and W2K mounted volume c:\mountpnt
;
JournaledFileSystems=c: d: c:\mountpnt
;
; Default journal db size
; Allows journal to grow, limited only by space available
JournalDBSize=0x00000000
;
; What events to monitor
;
; File & Dir Name, Attrib, Size, Security
; (no caching, immediate notification)
;NotifyFilter=0x00000107
;
;
; Notification Buffer Size
;
NotifyBufferSize=0x00200000
;
;
; Override settings for individual journaled filesystems.
;
; These settings override settings in JournaledFileSystemSettings
;
; Example of configuring a shared cluster resource override stanza
;
[JournaledFileSystemSettings.D:\]
JournalDBSize=0x0020000
NotifyFilter=0x00000107
DirNotifyBufferSize=0x00100000
NotifyBufferSize=0x00500000
JournalDir=d:\tsmjournal
; Don't delete the journal when a journaled file system goes offline
PreserveDBOnExit=1
; allow journals to move between cluster nodes
DeferFsMonStart=1
DeferRetryInterval=30
logFsErrors=0
;
;
Common Problems
Notification Buffer Overflow Errors
The mechanism used to monitor file system change activity requires
a pre-allocated buffer to store file change notification entries for each
file system actively being journaled..
The size of this buffer is specified by the NotifyBuffer configuration setting.
If change notifications arrive faster they can be processed by the file system
monitor portion of the journal service the notification buffer will overflow, and
the file system journal must be invalidated because the integrity of the journal
has been compromised since change activity has been missed.
Buffer overflows occur when the file system is flooded with a very large
volume of change activity in a short window in time.
For example, copying or deleting a large directory tree (large in terms
of number of objects and directory depth) may cause buffer overflow
errors to occur.
Increasing the size of the NotifyBuffer setting may eliminate this problem,
but increasing the size beyond 10 megabytes isn't recommended as it
increases memory utilization and degrades system performance.
Journal Based Backup is really only a viable solution for environments in which
the amount of file system activity is light to moderate, and that the activity is
somewhat well distributed.
Running applications which touch every file (or a very large percentage
of files) on the file system, or which flood the file system with changes
in a very short period of time (such as copying a very large directory
tree) may make journaling unusable.
Deleting files from a Windows command prompt
Files deleted from a Windows command prompt always generate change
notifications for long file names with the compressed DOS style 8.3 name.
This presents several problems to journal based backup.
Since the file has been deleted there is no method of obtaining the original
long name of the file.
Any previous backup of the file would have been under the long name, and since
there is no reliable method of deriving the original name from the compressed name,
the individual file can't be expired.
Since the individual file can't be expired, journal based backup must force a
non-journal based backup to be performed on the path portion of the file.
Another problem with deleting files from a command prompt may cause a journal
database to exceed the maximum supported size of 2 GB.
Using the Windows "rd" (or "rmdir") command to delete a large number of
files may cause the journal database to grow to its maximum size of 2 GB.
This problem seems to be very rare, but can occur if the files being
deleted have very similar names, or if the 8.3 names are very similar.
Both these problems may be circumvented by disabling NTFS 8.3 naming for long file
names.
This can be done by changing the NtfsDisable8dot3NameCreation value
from 0 to 1 in the following registry key:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem
A reboot is necessary to complete the change. Note that this change does
not eliminate existing 8.3 file names, but suppresses the automatic
creation of new 8.3 names.
Note that many legacy DOS and 16-bit Windows applications rely on 8.3 file
names. Disabling 8.3 file name creation may make new files with names
longer than 8.3 inaccessible to these legacy applications.
Installing Multiple Journal Services on the same machine (Windows)
Multiple journal service are installed by using the dsmcutil client utilitity.
Each instance must specify a unique service name and specify a unique
journal configuration file via the /JBBCONFILE dsmcutil option, and each
config file must specify a unique value of the JournalPipe setting.
Backup clients specifiy the specific journal service to connect to by using
the JournalPipe client option to point to a specific journal service instance.
Example of the JournalPipe setting/option:
in dsm.opt (client options file):
JournalPipe=\\.\pipe\journalService1
in tsmjbbd.ini (journal config file):
JournalPipe=\\.\pipe\journalService1
Example of installing multiple services via dsmcutil:
dsmcutil install journal /name:"TSM Journal Service 1" /jbbconfigfile:"f:\pete\journal1.ini"
TSM Windows NT Client Service Configuration Utility
Command Line Interface - Version 5, Release 4, Level 0.0
(C) Copyright IBM Corporation, 1990, 2007, All Rights Reserved.
Last Updated Jan 9 2007
TSM Api Verison 5.4.0
Command: Install TSM Client Service
Machine: BARKENSTEIN(Local Machine)
Installing TSM Client Service:
Machine : BARKENSTEIN
Service Name : TSM Journal Service 1
Client Directory : F:\tsm540c\debug\bin\winnt_unicode
Automatic Start : no
Logon Account : LocalSystem
The service was successfully installed.
Creating Registry Keys ...
Updated registry value 'ImagePath' .
Updated registry value 'EventMessageFile' .
Updated registry value 'TypesSupported' .
Updated registry value 'TSM Journal Service 1' .
Updated registry value 'ADSMClientKey' .
Updated registry value 'jbbConfigFile' .
Starting the 'TSM Journal Service 1' service ....
The service was successfully started.
Useful Tools and Utilities
The following tools may be obtained from the anonymous FTP server at ftp.software.ibm.com
(see exact paths below).
Journal Database Viewer (dbview.exe)
Display information about a specified journal database, list,
find, or delete journal datebase entries.
Note that the format of the journal database changed in TSM client version 5.3.0
and again in TSM 5.4.0, so different versions of the utility are required to view
journals created by those versions.
Syntax:
Usage: dbviewb JDBName [i]
JDBName is the fully qualified base name of a journal database file.
For example c:\tsmjournal\tsmF__.jdb is base name of the journal database
for drive F: .
Valid Option:
i - forces to enter interactive command mode
Valid commands in command mode:
List - List the entire contents of the journal database
Find - Find the journal database entry for a specified
object. Note the search is case sensitive.
Del - Delete a journal database entry for a specified
object.
Example:
D:\jbbutils>dbviewb c:\tsmjournal\tsmF__.jdb
Notification Filter Generator (qfilter.exe)
Display notification filter information in a
readable manner.
When a numeric notification filter is specified
the list of individual change activity flags contained
in the filter will be displayed.
When a space delimited list of change notification flags
is specified the numeric notification filter will be
displayed.
Note that the numeric filter value corresponds to the
JBB NotificationFilter configuration setting.
Syntax:
Usage: qfilter [numeric filter value | sym val1...]
Examples:
D:\jbbutils>qfilter 0x113
D:\jbbutils>qfilter fn dn lw se
These unsupported tools can be downloaded from:
V5.3x
/storage/tivoli-storage-management/patches/client/v5r3/Windows/unsupportedjbbutilities
Files:
-rw-rw---- 1 17037 620 98371 Feb 26 14:16 dbviewb.exe
-rw-rw---- 1 17037 620 90112 Feb 26 14:16 filemon.exe
-rw-rw---- 1 17037 620 49152 Feb 26 14:16 qfilter.exe
-rw-rw---- 1 17037 620 49152 Feb 26 14:16 volinfo.exe
V5.4+
/storage/tivoli-storage-management/patches/client/v5r4/Windows/unsupportedjbbutilities
Files:
-rw-rw---- 1 17037 620 102400 Feb 26 14:17 dbviewb.exe
-rw-rw---- 1 17037 620 90112 Feb 26 14:17 filemon.exe
-rw-rw---- 1 17037 620 49152 Feb 26 14:17 qfilter.exe
-rw-rw---- 1 17037 620 49152 Feb 26 14:18 volinfo.exe
JournalDir stanza configuration settings
Setting Platform Default Description
TraceFile All Trace.out Trace file
TraceFlags All None List of space delimited trace flags
TraceMax All None Maximum size of the aggregate trace file
TraceSegMax All None Maximum size of trace file segments
ErrorLog All Jbberror.log Fully qualified journal daemon error log
ErrorLog Max All None Maximum size of the error log before wrapping will be done
JournalDir All Directory where journal daemon resides Location of journal database, may be overridden for particular journals with override stanzas
JournalPipe All Depends on platform Default session manager pipe clients connect to.
Should only be changed when configuring multiple journal daemons on the same machine.
(Not allowed on AIX)
Corresponds to the client option of the same name.
SessionTimeout All 1200 Time in seconds a journal daemon client session will wait for a client response
JournaledFileSystems All none Space delimited list of file systems to monitor
JouraledFileSystem stanza configuration settings
Setting Platform Default Description
JournalDbSize All 0 (unlimited size) Size journal databases may grow to in bytes
notifyFilter Windows 0x117 Type of change activity to monitor
notifyBufferSize Windows 0x200000 (2 meg) Notify buffer size for non-directory name changes
dirNotifyBufferSize Windows 0x100000 (1 meg) Notify Buffer size for directory name changes
PreserveDbOnExit All 0 (disabled) Prevents journal from being deleted and reset when they are brought offline
deferFsMonStart All 0 (disabled) Allow file systems which aren’t available to be placed in defer mode
deferRetryInterval All 5 seconds Interval in seconds in which deferred file systems will be checked for availability
logFsMonErrors All 1 (enabled) Log error messages when file systems are not available
Windows Notify Filter Flag values Flag Value Changes to monitor
File Name 0x01 File creations, deletions, and renames
Directory Name ** 0x02 Directory creations, deletions, and renames
Attribute 0x04 Attribute changes including NTFS metadata
Size * 0x08 File size
Last Write timestamp * 0x10 Last modified timestamp
Last Access timestamp 0x20 Last access timestamp
Creation timestamp 0x40 Creation timestamp
NTFS Security 0x100 NTFS Security descriptor
* change notifications deferred until system write cache is flushed
** mutually exclusive of other flags
Cheers
Ryan Partington
Source: http://www-1.ibm.com/support/docview.wss?rs=663&uid=swg21155524
