Monday, April 4, 2016

How to troubleshoot SVN-Based Deployment Synchronizer issues in WSO2 products.

Observation

Recently when I was setting up a WSO2 API manager cluster which had publisher/store/key manager in a one node, gateway manager and gateway worker node having clustering and SVN-Based Deployment Synchronizer [1] enabled. The issue I faced was even though all my nodes started with no errors after publishing an API my API invocation failed with below error message in the carbon log of gateway worker node. 

[2015-09-16 17:36:29,279]  INFO - LogMediator STATUS = Message dispatched to the main sequence. Invalid URL., RESOURCE = /api1/1.0.0/customerservice/customers/123

Possible cause of the issue

  • This issue can occur when synapse configuration of the particular API is not in the /repository/deployment/server/synapse-configs/default/api synapse folder of the worker nodes. 

How to troubleshoot Deployment synchronisation issues 

1. Check /repository/deployment/server/synapse-configs/default/api folder for the API synapse config in each of the worker node.

If you cannot find the synapse config in the specified location there should something wrong with the deployment synchronisation.

As the next step you can check whether particular synapse file is in the SVN server repo. (URL of the SVN repo should be configured in 'DeploymentSynchronizer' section in _repository/conf/carbon.xml file in gateway worker nodes as well as gateway manager nodes.)

What configurations I should check?

Scenario 01 : Say you can see the synapse files in svn server location but not inside /repository/deployment/server/synapse-configs/default/api folder of the worker node

This means your manager node is successfully committing configurations into the SVN location yet worker nodes does not checkout the files.

This can happen due to incorrect configuration in 'DeploymentSynchronizer' section in  /repository/conf/carbon.xml file in gateway worker nodes. In the worker nodes make sure you have set 'AutoCommit' parameter to false and 'AutoCheckout' parameter to true.

Scenario 02 : Say configuration files are not committed to svn server location.

This means gateway manager node is not doing his work of committing artefact to the svn location.

This can happen due to incorrect configuration in 'DeploymentSynchronizer' section in /repository/conf/carbon.xml file in gateway manager node. In the manager node make sure you have set both  'AutoCommit' parameter and 'AutoCheckout' parameter to true.

For more information please refer to [1].

Note: 
Another possible reason for svn based deployment synchronization to not work is that your cluster is not configured properly. Normally when gateway manager commits an artifact to svn server location a cluster message is sent to worker nodes and once it is received workers check out the changes from the svn location. If everything is working fine, relevant messages should get printed on carbon logs of gateway manager and workers.

If you cannot see these messages check clustering configurations done under clustering section of the /repository/conf/axis2/axis2.xml file in all three nodes.  Verify that you have given the same domain name in all nodes and defined well known members correctly in the member section.

[1] https://docs.wso2.com/display/CLUSTER420/SVN-Based+Deployment+Synchronizer