Synchronization service technical description

Introduction

This document explains the synchronization process, the different job types, the difference between push synchronization and pull synchronization, and the mechanisms used in the communication with the service on other servers. Also covered, is the synchronization to NAS boxes.

The Basics

Like all other CapaInstaller services, the CapaInstaller Synchronization Service is located in the Service folder under the installation root. This is usually; C:\Program Files­\CapaInstaller­\Services­\Synchronization.

Along with the service, you will find the CISync Settings.xml which contains the settings for the service; Authentication method, SQL server name, Database name, port settings, and query interval for communication with the SQL server.

The CapaInstaller Synchronization Service does not, like most services, have a graphical interface. For that reason, the log file is the most important message interface available, when checking out the functionality for the service. In an effort to avoid being overwhelmed by information, the log activity is divided into more files.

The main logfile of CISync is CISync.log. It is placed in the Logs\Services folder under the installation root., usually; C:\Program Files­\CapaInstaller­\Logs­\Services. Previous logs are store in the \History subfolder. This is the log file in which initialization, job scheduling, and overall progress are covered.

For each Sync job, there exists a log file covering only that single job. It is located in the \logs­\Synchronization folder and is named; Sync_(Source server)_(Target server).(Job name).log, i.e. ; Sync_WS01.capasystems.com_ws09.capasystems.com.New09. Resources.log.

Job types

The Synchronization service is always installed on the Management point. On the Management server, however, it is a choice. Once installed, the service connects to the SQL server, to identify itself, and then checks for new jobs every 10 seconds.

The service is capable of handling jobs from two different sources.

  • Jobs to be run now, also called 'Update now' jobs.
  • Jobs that run by schedule.

Each of these two categories is handled similarly. When there is a new 'Update now' job available, or the schedule dictates, that a job is about to start, the service prepares the synchronization.
While preparing a job, the service distinguishes between package jobs and regular jobs. Package jobs are special in the sense that they represent the root of the container, rather than simple source and target paths for the synchronization.
Since packages are located in three different folders (ComputerJobs, UserJobs, and PocketPCJobs), and the number of packages to synchronize is variable, a package job actually spawns into x number of jobs corresponding to the number of packages. These package jobs, have their individual source root and target root, pointing directly to the sources and targets of the package.
An example of this is D:\Production\ComputerJobs\HWInventory\v4.1.
Jobs, that are not package jobs are sometimes also referred to as ‘SyncAll’ jobs. This is because they simply move data from the source to the target, without interpreting the paths.

Looking for a job

A synchronization job basically consists of three entities; A service ID, a source, and a target. In principle, the task of the Synchronization service is to keep the server updated from its parent. This means, that the source is a share on the parent Management server/point, and the target is a share of the server on which the service is installed. This is called a pull synchronization.
Nevertheless, we need to keep servers up to date that cannot run the service. This can be a workgroup server that cannot connect into the domain or a NAS box that cannot run windows compiled code. In such a situation the synchronization is carried out by the parent server, and instead of the Service ID pointing at the target server, it is the Service ID of the service running on the source. Because the Service copies data from itself onto the target, this is called a push synchronization.

Pull Synchronization

Pull synchronization is the most efficient synchronization method for more reasons. Primarily, it relies on the Sync service on the parent server (source) to perform certain tasks. Secondly, it utilizes both file compression to minimize the bandwidth needed and BITS to preserve available bandwidth to other tasks on the server.
Besides being efficient in transferring data between two servers, it is generally advisable to minimize file system requests. It can be a very lengthy process of requesting file info per file. Therefore the CISync Service asks the service on the source (called the proxy) to generate a list of the files on the source.
The service itself generates a similar list of the files on the target. The two lists are then compared, and the difference (if any) is then reported to the proxy. The proxy then creates a zip file consisting of the requested files.
When the zip file is generated the service establishes a BITS connection to the proxy, requesting the zip file. This process can take a long time.
In the logfile, these processes are shown graphically with 'growing' bars to illustrate the progress. The main reason for doing this, is to avoid interruption from the administrator (probably you), because you thought that no progress was made at all. The bars look like this;
Creating a compressed file: |...25%.
The line will grow over time, and will eventually look like this;
Creating compressed file: |...25%...50%...75%..100%|
To see the progress 'live' it is advisable to use an incremental viewer like Wintel
Upon completion, the zip file is unzipped, and the content is used to update the existing (and outdated) files

Push Synchronization

The push synchronization exists on more difficult terms. It cannot rely on BITS and because there is only one active part in push synchronization, zipping the content to be transmitted is obviously not feasible. While there is no counterpart to do the unzipping.
Furthermore, the CISync service needs to connect to the target server with encrypted credentials to establish an SMB connection directly to the target share
Partly because the CISync service probably runs in the Local system context and partly because we cannot rely on a specific brand of NAS boxes, establishing an SMB connection can be troublesome indeed.
For both the Workgroup server and the NAS box, a local user account is expected, and for exactly that reason the push synchronization has a party trick up its sleeve. The general idea is to let windows connect to the target (workgroup server or NAS box) with cached credentials. To achieve this CISync creates a local user on the server with the same username and password as required on the target.
Be aware that CISync cannot create a local user on a Domain controller. In that case, the account with the corresponding filesystem-rights must be created manually as a Domain User.