Integrating Web Service with Datastage

Web services expose operations of custom functions accessible to applications through internet or intranet. Web services are interoperable, reusable and follow strict standards and protocols to be followed for accessing them.

SOAP (Simple Object Access Protocol) is a specification for exchanging information between application and Web Service. Web service request and response are encoded in XML format.

This blog is to provide step-by-step instruction on invoking a web service through Datastage.

At the high level, following are the steps to be followed.

1. Identify and Save Web Service Definition 
2. Import Web Service Definition 
3. Configure Web Service Operation 
4. Configure Web Service Request 
5. Configure Web Service Response

Identify and Save Web Service Definition

Many web services are publicly available for free. In this example, let us try to access Country web service published through webservicex site.

Open URL http://www.webservicex.net/WS/WSDetails.aspx?WSID=17&CATID=7 in a web browser.

WSDL definition is listed on the XML frame. 

If you are using Firefox,

Right click on WSDL definition frame

Select “This Frame” option from popup menu

Select “Save frame as” option to locally save the file

If you are using Internet Explorer

          Click on WSDL definition frame

          Select File Menu and Click Save As option to save the file locally

Import Web Service Definition

This step will help to import the WSDL file saved in previous step into Datastage

• Login to Datastage Designer 
• Select Web Service Function Definitions in “Import” menu.

   

• It opens Web Services browser which lists all the definitions already available on the Datastage project
•  Click on “Metadata Importer” option

•  Click on “Open File” option and select the location of WSDL file saved earlier

• Web service explorer will lists the available web service operations
• Right click one of the service operation and select Import all operations

 Configure Web Service Operation

Next step is to create a Server Job to access “Country” Web Service through Web Services Client Stage.

• Login to Datastage Designer and Create a new Server Job 
• From Palette, Select Web Services Client from Real Time group 
• Place a sequential file stage as output

• Open Web Services Client stage properties
• Click on “Select Web Service Operation”

• Click on “Select Web Service Operation”
• Select “GetCountries” option. This web services does not have any input argument to be passed and hence Web Services Client stage is used for data retrieval.

 

• Select “GetCountries” option and click on “Select this item” link

If the Datastage server is running behind a proxy for internet access (as most of the site does), configure proxy settings in Proxy tab

• Enable HTTP/HTTPS option
• Supply user name and password for accessing internet 

• Specify proxy host name (without http). If host name and port details are not known, its a general practice to contact Datastage Administrator to obtain such information. 

If Datastage sever and Datastage client machine are directed through same internet proxy, these details can be extracted using the steps below. However, this is not recommended and demonstrated here for test purpose only.

Open Control Panel, Internet Options.

Select LAN Settings from Connections tab

Open the URL specified in LAN settings on a web browser

Host name and port details are generally specified at the “return” statement in configuration script. Supply this information in Web Services Client stage.

NOTE:

• This would work only when the server is already rightly configured for internet access through proxy 
• Include the server details in Web Client Stage excluding the file name

Configure Web Service Request

Open Web Services Client stage, Output tab

Open Input Arguments tab

Click on “Load Arguments Information” button. This loads namespace definitions into canvas

Configure Web Service Response

Open Web Services Client stage, Output tab

Open Output Message tab

Click on “Load Arguments Information” button. This loads output namespace definitions into canvas

Open Columns tab and check whether output column is populated 

Save, Compile and Run the job

Output file should have contents similar to the following

References

IBM Web Services Pack Guide