Azure Integration services for Dynamics 365

This blog describes the development best practices for the Azure Logic App.

Design best practices for Azure Logic App-based enterprise integration

The following are the design best practises for an enterprise Logic App integrations.

Loosely coupled integrating using Logic Apps

The loosely coupled architecture ensures that the producers (senders) and consumers (receivers) do not have to be sending and receiving messages at the same time, because messages are stored durably in the queue. Furthermore, the producer does not have to wait for a reply from the consumer in order to continue to process and send messages. We should decouple applications/systems from one another, this way the sending application does not need to “know” anything about the system(s) that should receive the data and is also not dependent on the receiving systems. The same applies to the applications that receive data. To make this a bit more specific, in general:

Source System will send the message to a service bus or to Logic App (via HTTP endpoint), which will enrich the data from the source system and then send the message to the service bus.
This message will trigger an Azure Logic app, which will validate, transform, enrich (to potentially a generic canonical data-model entity) and route the message to another services bus queue or topic.
Another “receiving-application specific Logic app” will be triggered and will pick this message up and do the receiving-application specific transformation. Then send the message to receiving application.

Azure Integration : Loosely coupled integrating using Logic Apps : Design development best practices for Azure Logic App

The benefits of Decoupled Architecture include:

Higher Performance: It will be possible to execute individual components independently, and in parallel.
Modularity: the ability to have separate components which are specific to the system which we are integrating.
Debugging: It is possible to track the issues at the separate module level and handle them separately.
Scalability: Any changes or improvements on the source or target system can be implemented easily.
Testing: Unit testing, integration, and performance will have separate workflows for each module. This makes testing more manageable and allows splitting it into smaller chunks, also making the process more effective.
Increased Agility: This modular model allows for more dynamics around feature release and upgrades. Only the targeted module needs to be upgraded at one time, leaving the rest of the application intact.

Reliable message processing while designing integration using Logic Apps

Azure Integration services communicats with different applications via connector using HTTP.An integration service has to ensure the guaranteed message delivery. It should also ensure that the messages reach the right services and their integrity and confidentiality are being maintained appropriately. In most of the usecases the integration should ensure the following in the message handling:
Once and only delivery
Ordered delivery

The central capability of a message broker such as Service Bus is to accept messages into a queue or topic and hold them available for later retrieval. When the broker transfers a message to a client, the broker and client want to establish an understanding of whether the message has been successfully processed and can therefore be removed, or whether the message delivery or processing failed, and thus the message might have to be delivered again. There are two types of delivery

ReceiveAndDelete
Peek-Lock Message

Make the Logic App idempotent

In the messaging architecture, the idempotency means making sure that it is possible to process the same message infinite times, and the result will always be identical. It means no duplicate database records, no emails, or other types of notifications sent many times. This approach has the following benefits:

Excludes the need for transactions. They can be quite tricky in the messaging infrastructure. Usually, message handlers call other services, and it is quite difficult to manage transactions across many systems.
Enables the use of retries without concerns that something might fail when the consumer processes the message the second time.

This can be achieved using Upsert actions in Logic App.

Design the Logic Apps for reuse

The enterprise integration involves common tasks which need to be executed over and over. These common tasks should be built as a simple logic app, which in turn can be invoked from other logic Apps. This will ensure the re-usability of the Logic Apps.

Secure the Logic Apps

The LogicApps with the HTTP Request Trigger is a publicly exposed endpoint, which can be accessed over the internet. These endpoints need to be secured, otherwise, anyone with the URL can post the information to the Logic APP and trigger the Logic App. By default, The endpoint is secured via a couple of mechanisms, including HTTPS and a SAS token. For Most organizations, there should be an additional form of security to meet their security needs. The following is the list of security options to secure the Logic App endpoints.

Access the endpoint using SharedAccess Token: Default
Restrict based IP
Authorization using Logic App
Logic Apps and APIM (Azure API Management)
Expose Logic App as API
Deploying Logic App in an isolated environment.

More info on securing Azure Logic App

Well built exception handling for Logic Apps

Retry policies for Logic App actions

A retry policy specifies whether and how the action or trigger retries a request when the original request times out or fails, which is any request that results in a 408, 429, or 5xx response.

Catch and handle exceptions by changing “run after” behavior

In each action definition, the runAfter property specifies the predecessor action that must first finish and the statuses permitted for that predecessor before the successor action can run. To make sure that an action can still run despite its predecessor’s status, set actions run after property to handle the predecessor’s unsuccessful statuses. Customize an action’s “run after” behavior so that the action runs when the predecessor’s status is either Succeeded, Failed, Skipped, TimedOut, or any of these statuses

Scopes to group actions in Logic App

Azure integration services : Exception Handling and Monitoring : Design development best practices for Azure Logic App

Monitoring the Logic Apps executions

There are times the Azure integration components such as Logic Apps, function Apps, fail due to technical errors, functional errors or data errors. There should be a mechanism to monitor and pro-actively inform the technical owners or functional owners about these errors, to have a successful and reliable integration. This blog provides a mechanism for the following errors:

Monitoring solutions
Handling Technical Errors
Tracking the integration flow errors

azure integration logging monitoring excepting handling : Azure Integration Logic App development best practises — azure integration logging monitoring excepting handling Azure Integration Dynamics 365 Logic App Function App

More info on monitoring and exception handling

Use the integration design patterns

Development best practises for Azure Logic App

Naming Convensions for Logic Apps

An effective naming convention composes resource names from important information about each resource. A well-chosen name helps you quickly identify the resource’s type, its associated workload, its deployment environment, and the Azure region hosting it. This would improve governance and maintainability.

Naming Resource Groups

The following naming convention can be used to name the resource groups in which the logic App will be created <resource Type>-<WorkLoad/Application/FunctionalDomain>-<Environment>-<instance>
e.g. rg-docMgmnt-Dev-001

Naming the Logic App

The following naming convention can be used to name the Logic Apps.
<resource Type>-<WorkLoad/Application/FunctionalDomain>-<SourceSystem>-<TargetSystem>-<environment>-<instance>
e.g. lapp-docMgmnt-http-ASB-dev-001.

Name the Logic App Actions

The Logic App name should be named descriptively. The name should describe the action it performs. This would improve the readability, helps with knowledge transfer, and improves maintainability. e.g. The action Get a row from Data verse should have the descriptive name as mentioned below.

Add Comments to Logic App actions

Comments are intended for anyone (including your future self) who may need to maintain, refactor, or extend the Logic App. The steps to add comments are : Click on … the Logic App Action > Add a comment > Then Add the comment

Add comments to the Logic App Actions : Azure Integration Logic App development best practises

Parameters in the Logic App

In Azure Logic Apps, use parameters to abstract values that might change between environments. Introducing parameters in Logic App, we can replace the environment-specific variables during the deployment. An example would be the URL of the D365 CE or FO environment, which varies per environment. Adding the parameters can be done from Workflow Designer.

These parameters can be created and referenced in the workflow designer. These parameter values can be set in the Azure Resource Manager (ARM) template and parameters files. These Parameters are defined and set at deployment. So, even if you need to only change one variable, you have to redeploy your logic app’s ARM template. The parameter types available are:

Array
Boolean
Float
Int
Object
Secure Object
Secure String
String

Retry policy for Logic App actions

For the most basic exception and error handling, use a retry policy in any action or trigger where supported. A retry policy specifies whether and how the action or trigger retries a request when the original request times out or fails, which is any request that results in a 408, 429, or 5xx response. Setting retry Policy

In the design view, select the ellipses (…) button, and then select Retry Policy.

Azure Integration : Retry policy for Logic App actions : Design development best practices for Azure Logic App

If no other retry policy is used, the default policy is used. Here are the retry policy types:

Type	Description
Default	This policy sends up to four retries at exponentially increasing intervals, which scale by 7.5 seconds but are capped between 5 and 45 seconds.
Exponential interval	This policy waits a random interval selected from an exponentially growing range before sending the next request.
Fixed interval	This policy waits the specified interval before sending the next request.
None	Don’t resend the request.

Configure Run After in Logic App Actions

When actions are added in the Logic App, it implicitly defines the order to use for executing those actions. After an action execution is completed, the actions are marked with a status such as Succeeded, Failed, Skipped, or TimedOut. In each action definition, the runAfter property specifies the predecessor action that must first finish and the statuses permitted for that predecessor before the successor action can run. By default, an action that you add in the designer runs only after the predecessor completes with Succeeded status.
When an action throws an unhandled error or exception, the action is marked Failed, and any successor action is marked Skipped. If this behavior happens for an action that has parallel branches, the Logic Apps engine follows the other branches to determine their completion statuses. For example, if a branch ends with a Skipped action, that branch’s completion status is based on that skipped action’s predecessor status. After the logic app run completes, the engine determines the entire run’s status by evaluating all the branch statuses. If any branch ends in failure, the entire logic app run is marked Failed.

To make sure that an action can still run despite its predecessor’s status, set actions run after property to handle the predecessor’s unsuccessful statuses. Customize an action’s “run after” behavior so that the action runs when the predecessor’s status is either Succeeded, Failed, Skipped, TimedOut, or any of these statuses. For example, to send a response message after processing the input request message has been Failed, rather than Succeeded, change the “run after” behavior by following either step:

In the design view, select the ellipses (…) button, and then select Configure run after.

Configure Run After settings in Logic App Actions : Design development best practices for Azure Logic App

The action shape shows the default status that’s required for the predecessor action, which is Try reading the account from CE and send it to ASB in this example:

Change the “run after” behavior to the status that you want, which is has failed, is skipped or has timed out in this example:

To specify that the action runs whether the predecessor action is marked as Failed, Skipped or TimedOut, select the other statuses:

The Configure run after can be used to implement Try Catch Finally implementation used in exception handling

The actions in Logic can be grouped under Scope control.The scopes are used when you want to logically group actions together, assess the scope’s aggregate status, and perform actions based on that status. After all the actions in a scope finish running, the scope itself gets its own status. To catch exceptions in a Failed scope and run actions that handle those errors, you can use the runAfter property for that Failed scope. That way, if any actions in the scope fail, and you use the runAfter property for that scope, you can create a single action to catch failures.

Although catching failures from a scope is useful, the context will help you understand exactly which actions failed plus any errors or status codes that were returned. The result() function returns the results from the top-level actions in a scoped action by accepting a single parameter, which is the scope’s name, and returning an array that contains the results from those first-level actions.

Use Scopes to group related actions : Design development best practices for Azure Logic App

The result() function returns the results from only the first-level actions and not from deeper nested actions such as switch or condition actions.

Name API Connections and Add Tags in Logic Apps

The API connections used in Logic Apps should be named accordingly and tag the API connections with the Logic Apps associated with it.

Cleanup Orphaned API connections

As part of Logic App development, when the Logic App actions such as Dataverse, Service Bus, Blob storage actions are added, the API connections resources are added in Azure. Over the time of Logic App life cycles, you would end up with unused API Connections which are no longer used by a Logic App. These Orphaned API connections should be identified and removed.
The following blog describes a PowerShell to identify and clean up orphaned API connections.

Logging

In the Azure portal, find and select your logic app.
On your logic app menu, under Monitoring, select Diagnostic settings > Add diagnostic setting. "Add diagnostic setting"”>
To create the setting, follow these steps:
1. Provide a name for the setting.
2. Select Send to Log Analytics.
3. For Subscription, select the Azure subscription that’s associated with your Log Analytics workspace.
4. For Log Analytics Workspace, select the workspace that you want to use.
5. Under log, select the WorkflowRuntime category, which specifies the event category that you want to record.
6. To select all metrics, under metric, select AllMetrics.
7. When you’re done, select Save.
For example:

Tracked Properties: Tracking Azure Integration Runs

Azure Logic Apps has some additional capabilities beyond the out-of-box capabilities for Tracking custom properties, these are called Tracked Properties. The tracked properties could be a vendor account in Dynamics 365 FO integration or an account number in Dynamics 365 CE integration, which would help us to follow the end to end integration flow. These additional capabilities are unlocked by enabling diagnostics for a logic app and publishing the data to Log Analytics.
In the Azure Logic App, from the Settings option within each action, there is a Tracked Properties setting where static or dynamic values can be included based upon Logic Apps Workflow Definition Language.

With a Log Analytics workspace configured, and diagnostics configured within our logic app, we will see diagnostic events emitted to Log Analytics. If the Tracked Properties are configured, then the next time we run our logic app, we will find our custom diagnostics included in our Log Analytics data.

Monitoring

Secuirng Input & output

Managed Identity