Troubleshooting notification e-mail server configuration settings when e-mail message profile tests fail.
Product
ExacqVision Client
Solution
If the profile test fails, review the exacqVision Verbose System Logs (Configuration > YOURSERVER > Systems Logs tab) for information about the cause. Pro Tip: Filter the displayed search results for entries relevant to notifications by typing “notifypi” in the filter box.
Here are some common causes including coresponding log entries.
Invalid E-mail Server Password
Invalid E-mail Server Username
Invalid E-mail Server Port
Invalid E-mail Server Security Type – STARTTLS
Invalid E-mail Server Security Type – Default
Invalid E-mail Server Outgoing Mail Server (SMTP) Address
At times is may be necessary to change the exacqVision Web Server Logging Level. By far the most common usage is changing from the default Error logging level to Debug and then back again.
Product
exacqVision Web Server
Solution
In order to assist with troubleshooting exacqVision Web Server issues it is sometimes useful to increase the verbosity of the log files by changing the log level. Once the necessary information has been gathered it is good practice to change the log level back to the default, Error Logging Level.
Change to Debug Logging Level
From Web Service Configuration
Navigate to Configuration, then Logging
Click the dropdown and change the Logging Level to Debug
Apply the change and restart Web Service
Return to the Landing Page
Force or wait for the issue to recur.
Change to normal (Error Logging Level)
From Web Service Configuration
Navigate to Configuration, then Logging
Click the dropdown and change the Logging level to Error
Servers are reporting Health Warnings in exacqVision Enterprise Manager. The event details in EM show “This result is based on an Attribute check on Disk /dev/csmi0,2” Note: the device may vary for example /dev/csmi0,0 instead of csmi0,2 or perhaps both.
The corresponding exacqVision Enterprise Manager logs show:
5/9/2021 02:09:11.356 PM (GMT-7:00) SysMgmtPI Warning 5516# Disk = /dev/csmi0,2 Status = This result is based on an Attribute check.
It has been determined that these are false positives and can safely be ignored as they do not affect normal operations. A fix will be available in exacqVision Server 21.09.
Video is not recording to customers iSCSI targets after a reboot. Drive was reporting a status of “Recording Not Possible” on exacqVision Server 21.03.10. Servers running exacqVision Server version 20.06 or older were showing a status of healthy, but were not recording.
Identification
Use the following to properly identify the issue before applying the fix.
Update to exacqVision Server version 21.03 or later.
Check the drives status on the storage page.
If the status is “Recording Not Possible”
Enable debug level logging
Restart the exacqVision Server service
Monitor for log entries similar to those listed below, specifically Store Fails and GRV Fails
Check Extended Storage or Enterprise Storage for conflicting entries like those displayed below.
Once positively identified follow the steps outlined in the resolution section
Log files (debug level enabled)
Before Fix
After Fix
Enterprise Storage page was showing the Type as Local (iSCSI) on the servers which were not recording. Servers which were recording report the volume type as Local. Note: The server showing the Status of “Recording Not Possible” was running newer exacqVision Server software 21.03.10 vs 20.06 or older for the others.
Another indication was that the iSCSI targets populated both the Targets and Partitions Section of the Storage page, Extended tab. Whereas the servers which were recording normally populated only the iSCSI Targets.section
Resolution
rename archivepi.xml
rename psfpi.xml
restart exacqVision Server Service
Verify that storage targets are only listed in the Partition section of the Storage page (Configuration > Storage > Extended Tab)
Verify that the volume now has a type of Local on the Enterprise Storage page (Configuration > Enterprise > Storage)
To assist in troubleshooting when drives are rejected for recording.
Change
Traditionally, the File Plugin (PSFPI) has logged ‘could not get record volume’ if no drives were found with space available. However, there are several scenarios that could lead to this result. Beginning in 9.6, the plugin will now provide additional logging to help understand the cause.
Here are some examples of the new log messages and their meanings:
PSFPI Warning 2 of 3 volumes are eligible for recording
This indicates the amount of drives checked for recording eligibility.
PSFPI Warning 2 bookmarks using 4.38GB
This shows how many files are bookmarked and cannot be deleted.
PSFPI Warning system has “At Least” rule configured for 30 days
This indicates if a system wide ‘At Least’ expiration rule is in place and for how long that video cannot be deleted.
PSFPI Warning stream 4720384 has “At Least” rule configured for 7 days PSFPI Warning stream 4720896 has “At Least” rule configured for 30 days
These indicate if per-device ‘At Least’ expiration rules are in place.
If drives are rejected for recording, the reason is indicated by the following:
PSFPI Warning volume C:\ rejected for recording: disabled
Drive has been manually disabled (un-checked) on the ‘Drive” tab.
PSFPI Warning volume D:\ rejected for recording: alarmed
Drive is currently in an alarmed state.
PSFPI Warning volume /mnt/edvr/2 rejected for recording: insufficient space: 1 GB wanted, 0GB available (reduced space from config slider: 30GB
This indicates that storage space is insufficient because files that cannot be deleted are consuming all available space; including space reserved by the ‘Video Space’ slider.
Once recording starts again, a log message will indicate which drive is now eligible.
PSFPI Verbose volume S:\ is now eligible for recording
It is possible for log files to become large enough to prevent proper access to an exacqVision system. If this happens, complete the following steps to delete the files manually:
Restart the system.
During the system startup process, press the Esc key to boot to shell.
