Using PeerGFS logs to troubleshoot
There are four primary logs to review when tracking down issues, in particular with individual files or folders. Three logs are stored in the PMC installation folder (typically C:\Program Files\Peer Software\Peer Management Center\Hub\workspace\logs) and one is stored in the Agent installation folder (typically C:\Program Files\Peer Software\Peer Agent\workspace\logs).
We recommend reviewing the logs in this order (most general to most detailed):
- fc_conflict.log
- fc_event.log
- fc_activity.log
- output.log (for PeerGFS v5.0 and earlier) or fc.log (for PeerGFS v5.1 and later)
There are additional logs that can be used for troubleshooting. Contact Peer Support for more information about these logs.
PMC logs
fc_conflict.log
This log contains a full history of all files that have been conflicted and/or quarantined, including:
- Optional to review this log; review only if you are investigating a conflict or quarantined file issue
- Notes on why the conflict and/or quarantine occurred
- Progress of a conflict towards becoming a quarantine
- How the conflict and/or quarantine was resolved
Sometimes, the quarantine reason in this file can point to an environmental or other "quickly" solvable issue (for example, Agent server ran out of disk space). If not, get the relative path to the file or folder of interest, and search for it in the fc_event log.
09-19-2019 12:58:21 261 1096 09-19-2019 12:58:21 Add \Bulk-4\C - Copy (3)\FILE16 - Copy (3).TXT File Quarantined File lock conflict --> Source Host: Lab12a-19a Conflict host: Lab12a-19a Windows error #3 : The system cannot find the path specified., path=\Bulk-4\C - Copy (3)\FILE16 - Copy (3).TXT File Lock 1591 24039
09-17-2019 11:50:46 284 1030 09-17-2019 11:23:23 Remove \bulk-1\A (48)\FILE16.TXT File Quarantined File Lock 9891 15882
fc_event.log
This log contains an overview of everything processed/detected by a session. Each entry includes:
- Timestamp (using the timezone of the PMC server)
- Severity
- Event code - see Event Log Details for information about the event codes. See fc_event_headers.xlsx for header information.
- Session ID
- Agent server where it occurred and notation on which was source and targets
- When applicable, user that owns the lock (Session represents the PeerGFS service account)
- Additional notes and/or messages
If any errors are listed in the fc_event log or something does not appear as expected, continue by checking the fc_activity log.
11-17-2022 13:38:28.347 INFO FO 456 Lab12A-AFS-16 Y 78424 \\AFS4\Marcus1\double\3 \FLDR33\FILEC19.TXT 1871 11-17-2022 13:38:17.225 moneypenny 11-17-2022 13:38:28.347 11-17-2022 13:38:28.347 5649 0 1400114 192.168.167.108
11-17-2022 13:38:28.414 INFO FL 456 Lab12A-Win-12R2 N 78374 E:\QASpotChecks\123\3 \FLDR33\FLDR33\FILEC19.TXT 3473 11-17-2022 13:34:57.000 Session 11-17-2022 13:38:28.294 11-17-2022 13:38:28.317 5649 DSU Stub File 23 1400052
11-17-2022 13:38:28.414 INFO FL 456 Lab12A-cDOT-12R2 N 78374 \\PTSVM93B\c$\PTSVM93b_cifs2\QASpotChecks\123\3 \FLDR33\FLDR33\FILEC19.TXT 3473 11-17-2022 13:34:57.622 Session 11-17-2022 13:38:28.294 11-17-2022 13:38:28.401 5649 107 1400052
11-17-2022 13:38:28.414 INFO FL 456 Lab12a-19a N 78374 E:\QASpotChecks\123\3 \FLDR33\FLDR33\FILEC19.TXT 3473 11-17-2022 13:34:57.000 Session 11-17-2022 13:38:28.294 11-17-2022 13:38:28.413 5649 DSU Stub File 119 1400052
11-17-2022 13:38:28.624 INFO FL 456 Lab12A-Win-12R2 N 78424 E:\QASpotChecks\123\3 \FLDR33\FILEC19.TXT 3239 11-17-2022 13:34:58.000 Session 11-17-2022 13:38:28.549 11-17-2022 13:38:28.581 5649 DSU Stub File 32 1400114
11-17-2022 13:38:28.624 INFO FL 456 Lab12a-19a N 78424 E:\QASpotChecks\123\3 \FLDR33\FILEC19.TXT 3239 11-17-2022 13:34:58.000 Session 11-17-2022 13:38:28.549 11-17-2022 13:38:28.617 5649 DSU Stub File 68 1400114
fc_activity.log
This log contains more in-depth information regarding the processing of each coordinator at the PMC. Each coordinator issues commands to Agents as necessary to accomplish a task (such as transferring a file from one Agent to another; or detecting an Open event on one Agent and locking that file on the other Agents).
11-17-2022 13:25:25.367 INFO 456 TID(165005) FileCloseCoordinator \FLDR33\FLDR33\FILEC19.TXT Operation Running in SLOW queue isExpeditedQueueProcessing=false isRunInSlowQueueOnly=false
11-17-2022 13:25:25.367 DEBUG 456 TID(165005) FileCloseCoordinator \FLDR33\FLDR33\FILEC19.TXT FileCloseCoordinator closeEvent=FAE Close - S Lab12A-AFS-16, moneypenny, eventId=1359420, \FLDR33\FLDR33\FILEC19.TXT, isDir=false, modTime=11-17-2022 13:25:00, size=3 KB, CW- (A:W+C+A,1180054), (S:R), SWA=false, (A), IT=true, HD=, Hdl=-1, SUCCESS affectTargets=false invalidAccessEvent=false scanUnsynchronizedFile=true hashCode=1689762063
11-17-2022 13:25:25.367 WARN 456 TID(165005) FileCloseCoordinator \FLDR33\FLDR33\FILEC19.TXT Not all configured hosts are participating. ParticipantDiffVO [hasAllConfiguredParticipants=false, isSourceNotParticipating=false, isTargetNotParticipating=false, FileTerminalParticipants [source=Lab12A-AFS-16, targets=Lab12a-19a,Lab12A-Win-12R2,Lab12A-cDOT-12R2]]
Agent logs
output.log - (For PeerGFS 5.0 and earlier)
This log contains only information regarding events that are detected on the given Agent or commands that are running on said Agent. All details and individual steps for these events and commands are logged to this file. Unfortunately, headers are only marginally useful; many log entries will have no formal column structure.
11-17-2022 12:38:41.055 ==========Callback appId=164, appSessionId=1844 QueueEventDataStructDebug [ event=WriteFile, userName=moneypenny, isFile=1, origPath=\\TestFS\SMB2\123\FLDR3\DATA39.TXT, newPath=null, eventInfo=, eventType=ChangeEvent, eventName=, accessLevel=0, sharedMode=0, eventNumber=16138776, eventTime=44882.52681209491, fileAttributes=0, isLocal=false, createDisposition=0, createOptions=0, threadProcCreated=false, processName=, synth=false, writeLength=0, writeOffset=0, remoteIP=192.168.167.108, shareName=, sessionId=1000, fileHandle=0, passCount=4034695, entryTime=0, msgStr=msgString not printed!] queueSize=77
11-17-2022 12:38:41.353 DCD.take()='FILE_MODIFIED, Lab12C-AFS-16, moneypenny, eventId=16138776, \FLDR3\DATA39.TXT, , 0 bytes, (?), , proc=' eventQueue.size=0 appId=164, appSessionId=1844
11-17-2022 12:38:41.353 new CD('\FLDR3\DATA39.TXT','FILE_MODIFIED, Lab12C-AFS-16, moneypenny, eventId=16138776, \FLDR3\DATA39.TXT, , 0 bytes, (?), , proc=') isSyncOnSaveEnabled=false isSyncOnSaveOverrideEnabled=false
11-17-2022 12:38:41.353 CD.submit('FILE_MODIFIED, Lab12C-AFS-16, moneypenny, eventId=16138776, \FLDR3\DATA39.TXT, , 0 bytes, (?), , proc=')
11-17-2022 12:38:41.353 CD.resume('\FLDR3\DATA39.TXT') containsCancelKey=true
fc.log - (For PeerGFS 5.1 and later)
This log generally contains the same information as output.log. However, some entries that aren't Agent-specific have been moved into separate logs. Contact Peer Technical Support if you need to review these logs.
11-17-2022 13:28:16.517 INFO WatchDirectoryOperations - ==========Callback appId=456, appSessionId=5649 QueueEventDataStructDebug [ event=WriteFile, userName=moneypenny, isFile=1, origPath=\\AFS4\Marcus1\double\3\FLDR33\FLDR31\FLDR33L1\FILEC9.TXT, newPath=null, eventInfo=, eventType=ChangeEvent, eventName=, accessLevel=0, sharedMode=0, eventNumber=766328, eventTime=44882.56126731481, fileAttributes=0, isLocal=false, createDisposition=0, createOptions=0, threadProcCreated=false, processName=, synth=false, writeLength=0, writeOffset=0, remoteIP=192.168.167.108, shareName=, sessionId=1030, fileHandle=0, passCount=4658, entryTime=0, msgStr=msgString not printed!] queueSize=120
11-17-2022 13:28:17.060 INFO DirectoryChangeDistillerWin32 - DCD.take()='FILE_MODIFIED, Lab12A-AFS-16, moneypenny, eventId=766328, \FLDR33\FLDR31\FLDR33L1\FILEC9.TXT, , 0 bytes, (?), , proc=, remoteIP=192.168.167.108' eventQueue.size=0 appId=456, appSessionId=5649
11-17-2022 13:28:17.060 INFO DirectoryChangeDistillerWin32 - new CD('\FLDR33\FLDR31\FLDR33L1\FILEC9.TXT','FILE_MODIFIED, Lab12A-AFS-16, moneypenny, eventId=766328, \FLDR33\FLDR31\FLDR33L1\FILEC9.TXT, , 0 bytes, (?), , proc=, remoteIP=192.168.167.108') isSyncOnSaveEnabled=false isSyncOnSaveOverrideEnabled=false
11-17-2022 13:28:17.060 INFO DirectoryChangeDistillerWin32 - CD.submit('FILE_MODIFIED, Lab12A-AFS-16, moneypenny, eventId=766328, \FLDR33\FLDR31\FLDR33L1\FILEC9.TXT, , 0 bytes, (?), , proc=, remoteIP=192.168.167.108')
11-17-2022 13:28:17.060 INFO DirectoryChangeDistillerWin32 - CD.resume('\FLDR33\FLDR31\FLDR33L1\FILEC9.TXT') containsCancelKey=true