Archiving from a Linux-based exacqVision system over a wide-area network (WAN) can stall in a Writing Files status for over an hour. If you believe a system is affected, verify the following conditions and symptoms:
- Linux kernel version 3.0 or later. Previous versions are not believed to be affected. (To determine the kernel version, run uname -r in the shell of the server.)
- The archive connection to the target is over a WAN in which disconnects are possible.
- The archive task status has not been updated in more than one hour. (In exacqVision 6.6 and later, the Status section on the Archiving page displays a State of “Failed — Target File Write Stalled.”)
- The currently archived file is not increasing in size. (To determine this, click on the details on the Archive page, note the Last Copied Content value, and navigate to the path of that content on the archive target. Verify whether the .workingps file is increasing in size periodically.)
<br>
This issue can be minimized by enabling strict caching mode and changing the cifs_max_pending cifs module parameter to 2 for Linux kernel 3.2 and later. To do this, complete the following steps:
- Open the /etc/fstab file on the server for editing.
- Locate the line with mount paths for the archive target.
- Add a flag to the comma-separated section of the line specifying other flags (normally, this should immediately follow rw). The flag added depends on the Linux kernel version used:
Linux kernel 3.2-3.7 (archiving performance will NOT be impacted if oplocks are supported on the archive target):
//10.16.2.4/disk1 /mnt/10.16.2.4.disk1 cifs _netdev,username=admin,password=admin256,rw,strictcache 0 0
Linux kernel 3.8 and later:
//10.16.2.4/disk1 /mnt/10.16.2.4.disk1 cifs _netdev,username=admin,password=admin256,rw,cache=strict 0 0
- Save the changes to the /etc/fstab file.
- Via shell terminal, execute umount [share path]. For example:
sudo umount /mnt/10.16.2.4.disk1 - Via shell terminal, execute sudo mount -a.
- Disable all archive targets (do not delete them).
- Stop using any other cifs mounts on the system.
- Unmount all cifs mounts using umount [path].
- Create a file named /etc/modprobe.d/cifs.conf with a line options cifs cifs_max_pending=2 (this persists the value across restarts).
- Unload the cifs module using sudo rmmod cifs.
- Reload cifs with a new parameter: sudo modprobe cifs cifs_max_pending=2.
- Verify the new value using cat /sys/module/cifs/parameters/cifs_max_pending.
- Enable the archive targets.
- Remount all other cifs shares.
<br>
If this procedure does not resolve the stalling issue, disable cifs client-side caching (this will impact archiving performance). You can try this in one of two ways:
- Open the /etc/fstab file on the server for editing.
- Locate the line with mount paths for the archive target.
- Add a flag to the comma-separated section of the line specifying other flags (normally, this should immediately follow rw). The flag added depends on the Linux kernel version used:
Linux kernel 3.2-3.7 (archiving performance will NOT be impacted if oplocks are supported on the archive target):
//10.16.2.4/disk1 /mnt/10.16.2.4.disk1 cifs _netdev,username=admin,password=admin256,rw,forcedirectio 0 0
Linux kernel 3.8 and later:
//10.16.2.4/disk1 /mnt/10.16.2.4.disk1 cifs _netdev,username=admin,password=admin256,rw,cache=none 0 0
<br>
Alternatively, you can enable strict caching mode on kernel 3.2 and later and also disable oplocks on the archive target. (NOTE: Strict caching mode is enabled automatically in exacqVision Server 6.6 and later.)
<br>
If you are using an exacqVision S-Series server (or other Samba-based server), oplocks can be disabled globally for all shares, or per share by adding the following lines to the appropriate section of the /etc/samba/smb.conf file:
oplocks=no
level2 oplocks=no
<br>
NOTE: If your environment’s configuration cannot tolerate the performance implications of disabled SMB caching (see above), you can use NFS protocol for archiving instead of SMB. Be aware that there is no username and password protection with the NFS option. Use this Knowledge Base article to configure NFS archiving.
