Thursday, October 2, 2014

Building a M3 Web Service over an API and SQL Server's FOR XML

As part of the series of posts on agile interface development with ION there are a couple of foundation aspects we need:

  • Building a M3 Web Service; and
  • Creating the XML file for for the Web Service

Building a M3 Web Service over APIs

I've covered building a M3 web service over M3 display screens before and building these over APIs is fundamentally the same, but to recap briefly there are two basic steps to this:

1. Create the Web Service in MWS Designer

Once you've determined the API transaction you want to use (in this example I'll use MMS025MI.AddAlias) build the web service in MWS designer by:

Creating a new web service

and specify the API transaction to use

Check the aliases on the fields for naming conflicts and resolve as required.  I've seen this as an issue with, for example, some Warning message fields where these often do not have unique aliases.
Once it's set up then deploy the web service.

This will have published the SOAP web service to your MWS service.  From here any SOAP compliant application can call the web service to interact with M3.

2. Validate the Web Service

From LCM, manage the MWS application.  From the List function you can then find your deployed web service.  Copy the http link for the default router.

To validate the web service works correctly and to get the exact format of the XML file required by the web service, I tend to use SOAPUI.  

In SOAPUI create a new web service

then manipulate the default request to reflect the call you want to make:
becomes

Execute the test and validate that the data has been posted into M3. 

The XML above is the template for the file we will create below.  We want everything apart from the <soapenv> tags so:

  <AddAlias>   
   <AddAliasItem>   
    <Company>100</Company>   
    <AliasCategory>2</AliasCategory>   
    <AliasQualifier>EA13</AliasQualifier>   
    <ItemNumber>  02010</ItemNumber>   
    <AliasNumber>4434567001143</AliasNumber>   
    <AliasType>EA13</AliasType>   
   </AddAliasItem>   
  </AddAlias>   


Note that when using SOAPUI, if your MWS is on the Grid then you'll use the authentication mechanism setup within LCM for this.  If your MWS is pre-Grid then you'll normally pass in the authentication as part of the XML string.  For this example I used MWS on the Grid which requires the authentication to be added via the Auth tab at the bottom of the SOAPUI screen.


Creating the XML file for for the Web Service

The starting point for this is a SQL Server table that has the field names and formats exactly the same as the Aliases in the M3 Web Service.  Note however that a date for M3 should be in the format YYYY-MM-DD.  It's better to use SQL type char(10) for this rather than datetime.

There are many articles showing how to use FOR XML to create XML output.  I won't reinvent the wheel here and will instead just jump straight to the solution we developed:

 declare @xml xml                    --Used to build the initial XML record  
 declare @output varchar(max);     --Used for the final XML record  
 --Obtain the inner data  
 WITH XMLNAMESPACES (   
 DEFAULT '""http://your.company.net/MMS025MI/AddAlias""')            
 select @xml = (  
 select   
 *  
 FROM [dbo].AddAlias AddAliasItem       
 FOR XML AUTO , ELEMENTS,root('AddAlias') )       
 --Fix the quotes around the namespace  
 select @output = (select convert(varchar(max),@xml))  
 set @output = REPLACE ( @output , '&quot;' , '"')  
 declare @cmd varchar(8000)  
 select @cmd = 'bcp "select ''' + @output + '''" queryout "C:\OUTPATH\data.xml"'  
      + ' -S SQLSERVERNAME -T -r -w -t -c -C RAW';  
 exec xp_cmdshell @cmd, NO_OUTPUT;  

Looking at the components of this:

 WITH XMLNAMESPACES (    
  DEFAULT '""http://your.company.net/MMS025MI/AddAlias""')  

This adds the XML namespace http://your.company.net/MMS025MI/AddAlias to the AddAlias node.

 select    
  *   
  FROM [dbo].AddAlias AddAliasItem  

This changes the node wrapping the actual data to AddAliasItem instead of AddAlias which is the table name.

 FOR XML AUTO , ELEMENTS,root('AddAlias') )   

This does two things.  Firstly by default FOR XML AUTO will treat the data as XML attributes on the AddAlias node.  Specifying ELEMENTS makes these appear as XML elements rather than attributes.  Secondly, while we renames the AddAlias node to AddAliasItem above, we still need an outer AddAlias node.  The command root('AddAlias') will add an outer AddAlias node wrapping the AddAliasItem node.

 select @output = (select convert(varchar(max),@xml))   
 set @output = REPLACE ( @output , '&quot;' , '"')  

The XML output from the FOR XML AUTO command will convert quote marks (") to &quot;.  ION however requires that quote marks appear as quote marks, so we use the SQL REPLACE command to find and correct these.

 select @cmd = 'bcp "select ''' + @output + '''" queryout "C:\OUTPATH\data.xml"'   
    + ' -S SQLSERVERNAME -T -r -w -t -c -C RAW';   
  exec xp_cmdshell @cmd, NO_OUTPUT;  

The final step is to output the generated XML data to the file system.  Here we use the SQL Server bulk copy command BCP to write the XML data to the disk.  We can call command line programs via xp_cmdshell and the NO_OUTPUT statement suppresses success or failure messages and makes the call cleaner.

This SQL logic will then take all records in the AddAlias and output these as a single XML file.  However, the M3 Web Service doesn't like multiple API commands in a single call, so we wrap this logic in a SQL cursor and output one record per XML file.  I'll cover the cursor logic in the last post in this series.

We can then take the XML file generated and test this in SOAP UI to confirm that what we have generated is compliant with the data required for the XML call.

Tuesday, September 9, 2014

Agile interface development using ION

I'm attending Inforum next week and will be presenting there on our experience using ION as part of an Agile interface development project.

In my presentation I cover briefly the details of the methodology we developed for this project which allows us to build a basic interface between two systems in about 30 minutes.  In this post I'll outline this methodology and, over the next few posts, I will build out the detail of how this works.

Note that the interface approach I'll describe here focuses on the plumbing between the two systems and intentionally moves business logic for any transformations, control flow or quality of service required outside the interface itself.


There are 5 main components of the interface:

  1. M3 Web Service
  2. ION Endpoint for Web Service
  3. ION Endpoint for file system
  4. ION Document flow to connect Web Service and file system
  5. SQL server stored procedure to create XML file for ION

Each of these components take about 5 minutes to set up, and the way they are set up is the same for all web services which makes for a very quick, repeatable process.

M3 Web Service

A M3 Web Service can be built over M3 APIs, most M3 programs, or SQL statements or stored procedures.  We've covered how to build web services over M3 programs and SQL statements before.  As part of this series I'll cover how to build a web service over an API.

Via M3 Web Services and the M3 Web Services Designer (both licensed modules from Infor) we expose the API as a SOAP web service, which can be widely consumed to allow easy integration with M3.  Conveniently, ION can consume these Web Services and thereby expose standard M3 APIs for List, Get, Create, Update and Delete functions.

ION Endpoint for Web Service

This advises ION the url that the web service will be exposed on, and the methods that will be exposed via this web service.  We also define the authentication with M3 (using a M3 username and password) and the identifier that will be used to track the processing of the ION message.

Through this endpoint ION will be able to call the web service we defined above.

ION Endpoint for File System

For this interface design we're accepting an XML file from an external system (in this case SQL Server) and passing this through ION to M3.  So here we define the source directory where ION will find the XML files.  This source directory can be on a network share, so we also specify the authentication to connect to that share.

Note that we could instead of using the file connector be querying a SQL Server directly via a stored procedure, but we chose the file system approach as it was ultimately simpler and led itself more to code reuse than the direct to SQL approach did.

ION Document flow

The document flow connects the endpoints together and the XML data that will be passed from the file system to the web service.  The document flow is where we can use the ION mapper to define business logic (like we can in MEC), but with this approach we do not.  ION is simply used to pass the message between systems.  Again this design decision reduced complexity and enabled reuse.

SQL Server to create XML file

Here we use SQL server to create the XML file that ION requires.  The SQL Server command "FOR XML" allows us to take the output from a SQL query and generate an XML document.  We then use bcp to create the XML file.



When we put these components together the architecture looks like this:

The two components on the left refer to the solutions we were integrating with M3 via ION, in this case Infor's Advanced Planner and Advanced Scheduler and a manufacturing plant.

In my presentation I go into some detail about the challenges inherent in those integrations and why an Agile approach was required.

The following posts will cover:
  • Building a M3 Web Service over an API and SQL Server's FOR XML
  • Building the endpoints and document flow in ION
  • Control logic and other considerations