This Blog describes the method of implementing Dynamics 365 UO’s Data Management Framework Recurring Integration Module using the Logic App. The blog provides a technical implementation of queuing and dequeuing of the Jobs using REST API of Dynamics 365 UO’s Recurring Integration Module. Dynamics 365 for Operation provides two primary sets of APIs, Recurring Integration APIs & Data Management Platform (DMF) package APIs, to support file-based integration scenarios. These APIs allow DMF data files as well as data projects to be imported and exported from Dynamics 365 for Operations.
In this use scenario for Dynamics 365UO recurring integrations, data will be exported to a downstream system on a recurring schedule using Logic App and store that information in Blob Storage. It shows how to export a data package and save it to a Blob and notify the systems using the Azure service bus.
More on Recurring Integration can be found here
Dynamics 365 UO Recurring Integration API for Export (dequeue)
To return a data package that contains all the data entities that were defined in the data project, and that the client application can unzip and consume, use the following structure.
https://<base URL>/api/connector/dequeue/<activity ID>
After the client downloads the data, an acknowledgment must be sent back to Finance and Operations, so that you can mark the data as received. In cases when there was no file uploaded to the blob, the dequeue API will return a response indicating as such.
- The execution Id of the DMF Job has been returned to the client application, which can be used to monitor the progress of the execution of the Job.
The set up involves the following set and the API supports import/export of DMF Data projects
Authorization for the Dynamics 365 UO Recurring Integration API
The integration REST API uses the same OAuth 2.0 authentication model as the other service endpoints. Before the integrating client application can consume this endpoint, you must create an application ID in Microsoft Azure Active Directory (Azure AD) and give it appropriate permission to the application. When you create and enable a recurring job, you’re prompted to enter the Azure AD application ID that will interact with that recurring job. Therefore, be sure to make a note of the application ID.
Set up a Dynamics 365 UO data project and Dynamics 365 UO recurring data jobs
Create a data project
- On the main dashboard, select the Data management tile to open the Data management workspace.
- Select the Import or Export tile to create a new data project.
- Enter a valid job name, data source, and entity name.
- Download a data file for one or more entities. Make sure that each entity is added, and that no errors occur.
- Target data format, select XML Element (Could use any format)
- Select Save.
Create a Dynamics 365 UO recurring data job
- On the Data project page, select Create a recurring data job.
- Enter a valid name and a description for the recurring data job.
- On the Set-up authorization policy tab, enter the application ID that was generated for your application, and mark it as enabled.
- Expand the Advanced options tab and specify either File or Data package.
- Select Set processing recurrence, and then, in the Define recurrence dialog box, set up a valid recurrence for your data job
- Select OK, and then select Yes in the confirmation message box.
Download data from Dynamics 365 UO using recurring data jobs
You can use integration REST endpoints to integrate with the client, submit documents (import), or poll available documents for download (export). These endpoints support OAuth.
Dequeue the Dynamics 365 UO recurring Export Job
Dynamics 365 UO recurring data jobs API for Export (dequeue)
Make an HTTP POST call against the following URL and In the message body, you can then pass the data as a memory stream.
| 1 | https://<base URL>/api/connector/dequeue/<activity ID> |
The following approach shows the way to dequeue the export Job to recurring integration. This approach uses data package-based export. Recurring integration supports both Data package export and the file export. The following parameters will be used
- The D365UO environment
- The Legal entity Name
- The ID of the recurring Job which was created in the previous step
- Name or description for the Export Job
Logic App and D365UO Export
The following section describes the method to create the Logic App for downloading Data Package from Recurring integration
- Sign in to Azure portal: https://portal.azure.com
- Select Create a resource, then select Logic Apps and click on Create
- Select the appropriate Resource Group and a Logic App Name and then click on Review+Create
- On the Logic App page select the Logic app designer option and then select Blank Logic App
- Search for Recurrence and select the Trigger.
- Select the Interval and the Frequency (based on your demand, shorter the interval, the more expensive the integration will be)
- Add Parameters for the Logic App
“ContainerName“: { “value”: “The Blob Container to store exported files” },
“Dyn365fOClientId“: {“value”: “Client Id user for the authentication” },
“Dyn365foURL“: { “value”: “The Url of D365FO environment xxxxxx.dynamics.com” },
“processQueueName“: { “value”: “The queue Name for The ExportJob” },
“ExportJobId“: { “value”: “The batch job execution ID” } - Add a New step, Search for Key Vault and Add an Action called “Get Secret”
- Add a Connection to the KeyVault, Enter the name of the Key Vault.
- Add a new step and search for Until
- Before you add value to control the loop, Add an action:
- Search for HTTP. In Actions select HTTP action [ This is the action that will execute the export Batch Job that we have created earlier. Provide appropriate Name for the Action]
- Fill in the Parameters
The URI to the DMF dequeue endpoint
Authentication Type: Active Directory oAuth
Authority: https://login.windows.net/
Tenant: Name of the tenant
Audience: Url of the D365UO environment (without the / in the end)
Client ID: The client ID registered in Azure AD and authorized in DMF Recurring integration)
Client Secret: The Application secret to connecting to Dynamics UO {The secret should be stored in KeyVault}
- Add a +New step and search for Condition
- Now you can add a value to control the loop. On ‘Choose a value’ text field and select the OutputParameters Status Code is not equal to 200
- Add a +New step and search for Until, Add the Until Action
- Search for HTTP. In Actions select HTTP action [ This is the action that will download the Data Package File. Provide appropriate Name for the Action]
- Fill in the Parameters
Method: Get
The URI would be the output from the Step 10 : replace(replace(body(‘HTTP’)[‘DownloadLocation’], ‘http:’, ‘https:’), ‘:80’, ‘:443’)
Authentication Type: Active Directory oAuth
Authority: https://login.windows.net/
Tenant: Name of the tenant
Audience: Url of the D365UO environment (without the / in the end)
Client ID: The client ID registered in Azure AD and authorized in DMF Recurring integration)
Client Secret: The Application secret to connecting to Dynamics UO {The secret should be stored in KeyVault}
- Now define the condition for Until Action. If the Value output parameters for the HTTP Status Code is equal to 200.
- The false condition indicates that the package can’t be downloaded. We will send an email with the message. Add an action and search for Send an email (V2):
- Sign in with your Outlook credentials and include the email information:
- Add an action for the True condition. Search for Create A Blob
- Add a connection to the Blob for the Create Blob connection
- Then Add New action to Send a Message to Azure Service Bus.
- Acknowledge The package download to Dynamics 365 UO Recurring integration using HTTP action. The HTTP code looks like below
{
"inputs": {
"method": "POST",
"uri": "@{concat('https://', parameters('Dyn365fOURL'), '/api/connector/ack/', encodeURIComponent(parameters('ExportJobId')))}",
"body": "@body('Get_the_Export_Job')",
"authentication": {
"audience": "@concat('https://', parameters('Dyn365fOURL'))",
"authority": "https://login.windows.net/",
"clientId": "@parameters('Dyn365fOClientId')",
"secret": "@parameters('Dyn365fOClientSecret')",
"tenant": "@parameters('Tenant')",
"type": "ActiveDirectoryOAuth"
}
}
}
- Save the Logic App. The Flow should look like this
- Run the Logic App
- Go back to the D365FO portal and review the Job history in the Data management workspace. A new export job with the name ExpensesExport should be executed
The JSON for the logic App is described below
{
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Condition": {
"actions": {
"Check_the_Download_is_successful": {
"actions": {
"Acknowledge_The_Package_Download": {
"inputs": {
"authentication": {
"audience": "@concat('https://', parameters('Dyn365fOURL'))",
"authority": "https://login.windows.net/",
"clientId": "@parameters('Dyn365fOClientId')",
"secret": "@body('The_Secret_for_D365UO')?['value']",
"tenant": "@parameters('Tenant')",
"type": "ActiveDirectoryOAuth"
},
"body": "@body('Get_the_Export_Job')",
"method": "POST",
"uri": "@{concat('https://', parameters('Dyn365fOURL'), '/api/connector/ack/', encodeURIComponent(parameters('ExportJobId')))}"
},
"runAfter": {
"Send_message": [
"Succeeded"
]
},
"type": "Http"
},
"Create_blob": {
"inputs": {
"body": "@body('Download_The_Package_File')",
"host": {
"connection": {
"name": "@parameters('$connections')['azureblob']['connectionId']"
}
},
"method": "post",
"path": "/datasets/default/files",
"queries": {
"folderPath": "/@{parameters('ContainerName')}",
"name": "@guid()"
}
},
"runAfter": {},
"type": "ApiConnection"
},
"Send_message": {
"inputs": {
"body": {
"ContentData": "@{base64(body('Create_blob')?['Path'])}",
"MessageId": "@{body('Create_blob')?['Path']}"
},
"host": {
"connection": {
"name": "@parameters('$connections')['servicebus']['connectionId']"
}
},
"method": "post",
"path": "/@{encodeURIComponent(parameters('processQueueName'))}/messages"
},
"runAfter": {
"Create_blob": [
"Succeeded"
]
},
"type": "ApiConnection"
}
},
"expression": "@equals(and(equals(outputs('Download_The_Package_File')?['StatusCode'], 200), not(equals(body('Download_The_Package_File'), parameters('EmptyFile')))), 200)",
"runAfter": {
"Wait_Until_The_Package_Is_Downloaded": [
"Succeeded"
]
},
"type": "If"
},
"Wait_Until_The_Package_Is_Downloaded": {
"actions": {
"Download_The_Package_File": {
"inputs": {
"authentication": {
"audience": "@concat('https://', parameters('Dyn365fOURL'))",
"authority": "https://login.windows.net/",
"clientId": "@parameters('Dyn365fOClientId')",
"secret": "@body('The_Secret_for_D365UO')?['value']",
"tenant": "heiway.net",
"type": "ActiveDirectoryOAuth"
},
"method": "GET",
"uri": "@{replace(replace(body('Get_the_Export_Job')['DownloadLocation'], 'http:', 'https:'), ':80', ':443')}"
},
"runAfter": {},
"type": "Http"
}
},
"expression": "@equals(outputs('Get_the_Export_Job')['statusCode'], 200)",
"limit": {
"count": 60,
"timeout": "PT1H"
},
"runAfter": {},
"type": "Until"
}
},
"else": {
"actions": {
"Terminate": {
"inputs": {
"runStatus": "Cancelled"
},
"runAfter": {},
"type": "Terminate"
}
}
},
"expression": "@equals(outputs('Get_the_Export_Job')['statusCode'], 200)",
"runAfter": {
"Wait_Until_Export_Job_Is_Complete": [
"Succeeded"
]
},
"type": "If"
},
"The_Secret_for_D365UO": {
"inputs": {
"host": {
"connection": {
"name": "@parameters('$connections')['keyvault_1']['connectionId']"
}
},
"method": "get",
"path": "/secrets/@{encodeURIComponent('\"SecretName\"')}/value"
},
"runAfter": {},
"type": "ApiConnection"
},
"Wait_Until_Export_Job_Is_Complete": {
"actions": {
"Get_the_Export_Job": {
"inputs": {
"authentication": {
"audience": "@concat('https://', parameters('Dyn365fOURL'))",
"authority": "https://login.windows.net/",
"clientId": "@parameters('Dyn365fOClientId')",
"secret": "@body('The_Secret_for_D365UO')?['value']",
"tenant": "@parameters('Tenant')",
"type": "ActiveDirectoryOAuth"
},
"method": "GET",
"uri": "@{concat('https://', parameters('Dyn365fOURL'), '/api/connector/dequeue/', encodeURIComponent(parameters('ExportJobId')))}"
},
"runAfter": {},
"type": "Http"
}
},
"expression": "@equals(outputs('Get_the_Export_Job')['statusCode'], 200)",
"limit": {
"count": 5,
"timeout": "PT1H"
},
"runAfter": {
"The_Secret_for_D365UO": [
"Succeeded"
]
},
"type": "Until"
}
},
"contentVersion": "1.0.0.0",
"outputs": {},
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
},
"Dyn365fOClientId": {
"type": "String"
},
"Dyn365fOURL": {
"type": "String"
},
"EmptyFile": {
"defaultValue": "",
"type": "String"
},
"ExportJobId": {
"defaultValue": "{18FA7FDE-557F-4421-BF6C-2C4417963C69}",
"type": "String"
},
"Tenant": {
"defaultValue": "heiway.net",
"type": "String"
},
"processQueueName": {
"type": "String"
}
},
"triggers": {
"Recurrence": {
"recurrence": {
"frequency": "Minute",
"interval": 2
},
"runtimeConfiguration": {
"concurrency": {
"runs": 1
}
},
"type": "Recurrence"
}
}
},
"parameters": {
"$connections": {
"value": {
"azureblob": {
"connectionId": "/subscriptions/9ce707d5-a33e-4243-ba8e-9a3826893136/resourcegroups/RG-Tasubaki-Encoding/providers/Microsoft.Web/connections/azureblob",
"connectionName": "azureblob",
"id": "/subscriptions/9ce707d5-a33e-4243-ba8e-9a3826893136/providers/Microsoft.Web/locations/westeurope/managedApis/azureblob"
},
"keyvault_1": {
"connectionId": "/subscriptions/9ce707d5-a33e-4243-ba8e-9a3826893136/resourceGroups/RG-Tasubaki-Encoding/providers/Microsoft.Web/connections/keyvault-2",
"connectionName": "keyvault-2",
"id": "/subscriptions/9ce707d5-a33e-4243-ba8e-9a3826893136/providers/Microsoft.Web/locations/westeurope/managedApis/keyvault"
},
"servicebus": {
"connectionId": "/subscriptions/9ce707d5-a33e-4243-ba8e-9a3826893136/resourcegroups/RG-Tasubaki-Encoding/providers/Microsoft.Web/connections/servicebus",
"connectionName": "servicebus",
"id": "/subscriptions/9ce707d5-a33e-4243-ba8e-9a3826893136/providers/Microsoft.Web/locations/westeurope/managedApis/servicebus"
}
}
},
"Dyn365fOClientId": {
"value": "0ae2fd1b-7b98-457c-b1c5-108824f924ec"
},
"Dyn365fOURL": {
"value": "nl1hhs-acc.sandbox.operations.dynamics.com"
},
"processQueueName": {
"value": "ExportJob"
}
}
}
- D365 FO introduces New user-based service protection API limits
- D365 FO: Priority based throttling for integrations
- Monitoring and alerting for Azure Key Vault
- D365 FO: Set financial dimension value using oData
- Azure Integration using Managed Identity
- D365 Finance and Operations integration using BYOD
Azure Integration services for Dynamics 365 
Good readding
Hi, I implemented rest API method and it was working fine. picking the correct REST API next batch from D365 and downloading data.
now there is one new batch, but it is picking old processed data… Not picking the latest data…
Please help me on this.