Exacq Support has identified an issue with Enterprise Manager importer processes using an excessive amount of memory resource, often requiring reboot due to system becoming unresponsive when installed on some Windows machines. This is estimated to affect versions 23.06.2.0 thru latest versions of ExacqVision Enterprise Manager for Windows. Developers are aware and working on resolving these issues.
NOTE: This high memory resource usage often corresponds with errors seen in importer log exceptions. You can trace the PID this process starts up on in importer-error logs to the process in task manager in Windows.
Product
ExacqVision Enterprise Manager
23.06.2.0 – 24.06.2.0
Solution
Engineering has developed a temporary workaround consisting of a PowerShell script developed to scan for and kill certain importer processes when their memory usage is at unacceptable levels. This script runs every 5 minutes, unless code is altered. This script should only be used as a last resort for Support personnel to deploy when all troubleshooting and other remediation steps have been exhausted. In some cases it may be recommended to alter the MemoryCeiling function of this script to better suite certain customer scenarios.
It is also recommended that this script be triggered as a scheduled task from task scheduler. This often requires that Execution Policies be bypassed to launch successfully on reboot. Detailed instructions are below:
2) Unzip the file and place the importerMonitor.ps1 file in a directory. Often times, it is best to place this in: “%\Program Files\exacqvision\”
3) Open Task Scheduler in windows and Create Task, then name task importerMonitor and configure an admin user and to run whether user is logged on or not. Be sure to check the box for Run with highest privileges, and configure for OS in use.
4) Under triggers, select New… Then configure for “At Startup”.
5) Under the Actions tab, select New… and configure as follows: – Start a Program – Program/script: Powershell.exe – Add arguments: -ExecutionPolicy Bypass – File “C:\Program Files\exacqVision\importerMonitor.ps1”
6) Configure the conditions tab as pictured:
7) Configure the Settings tab as pictured:
8) Reboot to test the Task at startup, and verify it worked as expected by observing the History tab in Task Scheduler for this task:
Enterprise Manager may fail to send email notifications after update to 23.09.9.0.
ExacqVision support has identified an issue with failing email notifications after updating Enterprise Manager from a previous version to version 23.09.9.0 and up. This occurs when checking “Anonymous” as the send user, or additional configuration required for Exchange servers has not been completed.
NOTE: Email notifications have been confirmed to succeed using same SMTP configuration on version 23.06.2.0 but fails when update to 23.09.9.0 is completed.
Product
exacqVision Enterprise Manager Version 23.09.9.0
Steps to Reproduce
Update Enterprise Manager to version 23.09.9.0 from a previous version
Expected Results
Email notifications should continue to succeed with existing configuration.
Actual Results
Email notifications may fail with all or one of the following log errors: Failed email verification: SMTP AUTH extension not supported by server.
Failed email verification: (550, b'5.7.60 SMTP; Client does not have permissions to send as this sender')
Failed email verification: STARTTLS extension not supported by server.
disable “anonymous users” as send from address in Enterprise Manager and replace with an authorized SMTP user.
2. Turn on Basic Authentication method in Exchange/ SMTP server if option is available.
3. Make sure the user account sending notifications from Enterprise Manager has SMTP Authentication turned on in their exchange profile, and also that the “Send From” address used in Enterprise Manager was added to the exchange user as an email alias.
Support has identified an issue with updating Enterprise Manager to version 23.09.5.0 prior to running updates to ExacqVision server 23.09.6.0. This issue will manifest after updating Enterprise Manager by users experiencing disconnection from the client with errors ” log in failed / invalid username or password or account locked or disabled” or ” log in failed – invalid response”.
Product
ExacqVision Enterprise Manager
ExacqVision Server
Steps to Reproduce
Update Enterprise Manager to version 23.09.5.0 while leaving ExacqVision server versions on 23.06 or lower.
Expected Results
– Users should remain logged in successfully.
Actual Results
– Users will be disconnected from the Client with error – ” log in failed / invalid username or password or account locked or disabled” or ” log in failed – invalid response”
Solution
Engineering is currently working on a patch to resolve this issue.
For customers that updated from 23.06 to 23.09 prior to the patch, installing the version with the patch should fix all login issues customers are facing.
For customers that performed a fresh install of 23.09 prior to the patch, user server passwords will become scrambled and customers will need to change/reset user passwords.
Until the patch is released, if customer has an Enterprise Manager backup that contains no issues from 23.06 that can be applied, the following steps can be taken:
Take a backup from 23.09 prior to making any changes.
Access a recent prior backup off of Enterprise Manager (this must be taken on version 23.06 previously).
Completely remove Enterprise Manager.
Install Enterprise Manager previous version 23.06.0.0.
Follow on screen prompts during install.
Login to Enterprise Manager as the root admin user.
Upload previously taken backup using the upload button in the backups screen in Enterprise Manager user interface.
Wait for this to complete.
Restore from uploaded backup.
Wait for restoration completion.
Confirm log-in capability has been restored.
Re-setup scheduled backups as these will be defaulted.
It should be noted that downgrading is not a normal solution, and should be prevented with Enterprise Manager. These instructions are only for situations where the customer has already performed this update and is now locked out of Client software and critically needs to be restored in a timely manner.
IMPORTANT For Instructions on current versions of exacqVision Enterprise Manager versions 22.06 or higher see Knowledge Base Article #12804
This document details how to enable HTTPS connections to exacqVision Enterprise System Manager on versions 22.03 or lower.
For a trusted certificate, it is recommended that you purchase a third-party intermediate certificate from one of many online providers. If you are using a third-party certificate you may skip ahead to the section titled, “Obtaining a Third-Party Certificate”.
These steps will detail how to create a self-signed certificate, but be aware that web browsers will warn users that the certificate is untrusted if you are using a self-signed certificate or one from a private/internal certificate authority.
CREATING A SELF-SIGNED SSL CERTIFICATE
Windows
1) Click on the Windows Start button and type ‘CMD’. Right-click on the CMD icon and choose ‘Run as Administrator’.
2) Set the environmental variable that will be used by OpenSSL later by typing:
set OPENSSL_CONF=C:\Program Files\exacqVision\EnterpriseManager\apache_solr\apache2\conf\openssl.cnf
Press Enter.
3) Change your working directory by typing:
cd "C:\Program Files\exacqVision\EnterpriseManager\apache_solr\apache2\bin"
You will be prompted to enter a PEM pass phrase. Enter anything you like but you will need to re-enter this in the following steps.
PEM pass phrase:
5) You will be prompted with several questions for the certificate, answer these according to your needs. COMMON NAME should be the IP address or FQDN that users will access to reach the ESM web site (ex. www.domain.com or esmserver.domain.com).
You will be prompted with a series of questions. – Use data specific to your site. – Items can be left blank with the exception of Common Name – Common Name (e.g. server FQDN or YOUR name) should be the IP address of EM Server
Verify the md5 hashes match, if they DO NOT then see the troubleshooting section below before proceeding.
Step 3 Edit Apache Configuration
cd /usr/local/exacq/esm/apache_solr/apache2/conf/extra
sudo gedit httpd-ssl.conf
Make the following changes, save the file and then close gedit.
Step 4 Restart the enterprise-webservice
sudo service enterprise-webservice stop
sudo service enterprise-webservice start
<br>
OBTAINING A THIRD-PARTY CERTIFICATE
If you are planning to acquire a third-party certificate from a trusted provider, you may need to provide them with a Certificate Signing Request (CSR) file.
Enter all the fields click on the ‘Submit’ button to download the ZIP file. Inside this ZIP file is the CSR file and RSA key to give to your certificate provider.
If you purchased a chained certificate, be sure to download the appropriate intermediate bundle.
Once you have downloaded the files from your provider:
Rename the .crt file to ‘server.crt’.
Rename the .key file to ‘server.key’.
If you have a chained certificate, rename the chain file to ‘server-ca.crt’.
Place the renamed files from your Certificate Authority (CA) into the following directory:
When purchasing an SSL certificate, many providers offer an Intermediate Bundle, or additional certificates that must be present to link your certificate to a root certification authority. Usually the provider will have documentation on how to accomplish this with Apache, but it is a good idea to ask them before or during the purchasing process. Exacq is not responsible for making your certificates capable of working with Apache.
It is possible to combine all the intermediate certificates that a provider may give you into one file. Consult your provider for more information.
<br>
ENABLING SSL FOR HTTPS CONNECTIONS
Be sure that you have followed the steps above to place the certificate files necessary for either a third-party certificate or a self-signed certificate into the correct directory before continuing with the following steps.
Windows
1) Click on the Windows Start menu and find the Windows Notepad program. Right-click on this and choose to ‘Run as Administrator’. If you do not run Notepad as an administrator you will be unable to save your changes.
2) With Notepad open, click on the ‘File’ menu and choose ‘Open’ or press CTRL-O on the keyboard.
In the Open browser, change the drop-down menu for File Type from ‘Text Documents (*.txt)’ to ‘All Files (*.*)’.
Use the Open browser to open the C:\Program Files\exacqVision\EnterpriseManager\apache_solr\apache2\conf directory and highlight the file titled ‘httpd.conf’ then click ‘Open’.
3) Find the following line:
LoadModule ssl_module modules/mod_ssl.so
Remove any pound (#) sign in front of this line if there is one.
Now, find the following line:
Include conf/extra/httpd-ssl.conf
Remove any pound (#) sign in front of this line if there is one.
Save the file.
4) Still using Notepad, open the file titled ‘httpd-ssl.conf’ located in C:\Program Files\exacqVision\EnterpriseManager\apache_solr\apache2\conf\extra
Find the following line:
ServerName www.example.com:443
Change the ‘www.example.com’ portion of this line to ‘localhost’.
Save the file and close the window.
5) Restart the solrApache service in Windows services (services.msc).
Linux
1) Open a Terminal prompt.
2) Change your working directory by typing:
cd /usr/local/exacq/esm/apache_solr/apache2/conf
Press Enter.
3) You may use any editor you feel comfortable with, such as vi or nano, but if your are more inclined to using a graphical interface you may use a program called ‘gedit’ to make the following changes.
In the Terminal, type:
sudo gedit httpd.conf
Press Enter.
4) Find the following line:
LoadModule ssl_module modules/mod_ssl.so
Remove any pound (#) sign in front of the line if there is one.
Now, find the following line:
Include conf/extra/httpd-ssl.conf
Remove any pound (#) sign in front of the line if there is one.
Save the file and close the ‘gedit’ editor window to return to the Terminal prompt.
5) In the Terminal, type:
sudo gedit extra/httpd-ssl.conf
Press Enter.
Find the following line:
ServerName www.example.com:443
Change the ‘www.example.com’ portion of this line to ‘localhost’.
Save the file and close the window to return to the Terminal prompt.
6) Restart the service in the Terminal by typing:
sudo service ESMWebservice restart
<br>
FORCED REDIRECT FROM HTTP TO HTTPS
If you want to force users who try to access the site on port 80, using HTTP, to use the secure HTTPS connection you will need to enable a redirection.
Windows
1) Click on the Windows Start menu and find the Windows Notepad program. Right-click on this and choose to ‘Run as Administrator’. If you do not run Notepad as an administrator you will be unable to save your changes.
2) With Notepad open, click on the ‘File’ menu and choose ‘Open’ or press CTRL-O on the keyboard.
In the Open browser, change the drop-down menu for File Type from ‘Text Documents (*.txt)’ to ‘All Files (*.*)’.
Use the Open browser to open the C:\Program Files\exacqVision\EnterpriseManager\apache_solr\apache2\conf directory and highlight the file titled ‘httpd.conf’ then click ‘Open’.
Remove the pound (#) signs in front of these two lines.
Save the file.
4) Restart the solrApache service in Windows services (services.msc).
Linux
1) You may use any editor you feel comfortable with, such as vi or nano, but if your are more inclined to using a graphical interface you may use a program called ‘gedit’ to make the following changes.
Remove the pound (#) signs in front of these two lines.
Save the file and close the ‘gedit’ window to return to the Terminal prompt.
3) Restart the service in Terminal by typing:
sudo service ESMWebservice restart or sudo service enterprise-webservice restart
<br>
TROUBLESHOOTING
1) Some versions of Internet Explorer do not easily work with services running locally or may display pages incorrectly. If this happens, try clearing the browser’s cache by pressing CTRL-F5 on the keyboard. If the problem is persistent try installing another web browser, such as Chrome.
2) If the solrApache service fails to start after configuring it for SSL:
[Wed Mar 04 09:08:54.512004 2017] [ssl:emerg] [pid 19116] AH02565: Certificate and private key www.example.com:443:0 from server.crt and server.key do not match AH00016: Configuration Failed
c) If you see this log entry, complete the following steps:
1) Change your working directory to the location of openssl.exe
Windows (CMD) – cd C:\Program Files\exacqVision\EnterpriseManager\apache_solr\apache2\bin
3) Compare the resulting values output after running each of the preceding commands. Each resulting string should be identical. If the values do not match, confer with the certificate authority that issued the certificate.
The exacqVision Enterprise Manager Server List page is showing a status of “Not Monitoring”
Product:
exacqVision Enterprise Manager
Steps to Reproduce
From exacqVision Enterprise Manager, choose Servers
A list of the added servers will be displayed on the Server List page
Expected Results:
Status of all listed servers shows “Connected”
Actual Results:
Some or all of the listed servers show “Not Monitoring”
Solution:
Use the exacqVision Client to check the license status for the the servers showing “Not Monitoring” to make sure that they are “Enterprise”. See Required Licensing specifications listed in the exacqVision Enterprise Manager Data Sheet. A new Enterprise License will be required to monitor these systems
Using the Exacq Software Updates Packaging Utility
Description
Systems which are not connected to the internet can be updated using the exacqVision Software Updates Packaging Utility in conjunction with exacqVision Enterprise Manager to deploy software updates (Server, Client and Web Service) throughout the VMS infrastructure.
Choose either Upload License File or Recent Software Versions
Upload License File
The advantage of using a license file is it will provide relevant versions if any of your systems have an expired SSA “Software Subscription Agreement”. Older versions of exacqVision Server are not available when Recent Software Versions is selected. Note: A CSV file must be created for each NVR, follow the steps displayed.
Click Upload License File, a file explorer will appear
Navigate to, and select the license.csv created earlier, click Open, the file will upload and the file explorer will close
The license.csv will be processed and the Software Selection window will appear
See Software Selection Window to continue
Recent Software Versions
A list of all the most recent versions of exacqVision client, server, and webserver software for various platforms.
Click Recent Software Versions, the Software Selection window will appear
See Software Selection Window to continue
Software Selection Window
Select each software package needed
Click Zip Selected Files
In a few moments a “Zipping Files” icon will appear
When complete a notification that the zip is now available for download will appear
WARNING: Depending on the number of packages selected, site traffic, and various other factors it can take a LONG time before the package will be available.
Click the Click to Download Packaged Files hyperlink to download the packaged updates zip file.
Transfer the zip file to the EM Server
exacqVision Enterprise Manager
exacqVision Enterprise Manager can be used to manage the exacqVision Server, exacqVision Client and exacqVision Web Server version software on NVRs or Web Servers.
From the exacqVision Enterprise Manager Dashboard
Navigate to Available Versions
Hover over the refresh icon , select Refresh from Zip File
A file manager window will appear, navigate to, and select the saved zip file, click Open
The file manager window will close and the “Please wait, updating versions list” banner will appear
When complete the “New installers are available” banner will appear
The versions selected in the Software Selection Window earlier should now be present as a Recommended Version.