D365 FO introduces New user-based service protection API limits

To ensure the consistent availability, reliability, and performance of the Finance and Operations service protecting against extraordinary usage, Microsoft will be introducing user-based service protection API limits. In version 10.0.32, with 2023 release wave 1, the API limits will be mandatory and the option to disable API limits will no longer be provided.

Why user-based service protection API limits for D365FO?

New user-based service protection API limits will soon be applied in D365FO environments to ensure consistent availability and performance of the service. 
These limits are designed to preserve service health in the event of client applications with extraordinary demands on server resources. Additionally, these limits are designed to throttle incoming API requests when there are sudden bursts of traffic or concurrent long-running requests against the server. 
The limits are in addition to the resource-based service protection API limits that have been in place for Finance and Operations apps environments since version 10.0.19 (which protects the environment from aggregated user demand exhausting environment resources). The user-based limits define specific API usage thresholds for OData and custom API endpoints to prevent individual users or integrations from causing outages or degrading environment performance.

What are user-based service protection API limits and how does it work in D365FO?

There are two types of service protection API limits for D365FO: user-based and resource-based.

  • User-based limits help prevent individual users or integrations from harming system performance and availability.
  • Resource-based limits help protect the environment by enforcing thresholds of high environment resource utilization. When high thresholds are reached, service requests are limited. More details

User-based service protection API limits

User-based service protection API limits are enforced based on three factors:

  • The number of requests that a user sent
  • The combined execution time that is required to process the requests that a user sent
  • The number of concurrent requests that a user sent

The first Two of the user-based service protection API limits are evaluated within a sliding window of five minutes (300 seconds). If either limit is exceeded during the preceding 300 seconds, a service protection API limit error will be returned on subsequent requests to help protect the service.

The following table describes the default user-based service protection API limits that are enforced per user, per application ID, and per web server.

MeasureDescriptionService protection limit
Number of requestsThe cumulative number of requests that the user has made.6,000 within the five-minute sliding window
Execution timeThe combined execution time of all requests that the user has made.20 minutes (1,200 seconds) within the five-minute sliding window
Number of concurrent requestsThe number of concurrent requests that the user has made.52

Number of requests

This limit counts the total number of requests during the preceding 300-second period. The following error message is returned together with the API response:

The number of requests exceeded the limit of 6000 over the time window of 300 seconds.

D365 FO user-based service protection API Number of request limits

Execution time

This limit tracks the combined execution time of incoming requests during the preceding 300-second period. The following error message is returned together with the API response:

The combined execution time of incoming requests exceeded the limit of 1200 seconds over the time window of 300 seconds. Decrease the number of concurrent requests or reduce the duration of requests and try again later.

D365 FO user-based service protection API Execution Time limits

Number of concurrent requests

This limit tracks the number of concurrent requests for the user. The following error message is returned together with the API response:

The number of concurrent requests exceeded the limit of 52.

D365 FO user-based service protection API Number of Cuncurrent request limits

How do monitor the throttled exceptions?

To see whether there are already integrations being throttled, the “Environment monitoring” in LCS offers insights into these “Throttling events”. This enables you to see which integrations are being throttled as we speak.

The D365FO logs all the throttled exceptions and can be retrieved using LCS.

  1. Log into LCS: https://lcs.dynamics.com/
  2. Select the correct Project
  3. Select the Environment such as PRD, ACC
  4. Then scroll down to “Monitoring” and click on “Environment Monitoring”
  5. Then click on Activity, and select the “Requests throttled” query. Then select the range of date to see the throttled requests.
D365 FO user-based service protection API Monitoring

Exceptions to service protection limits

The service protection API limits don’t apply to all Microsoft services. The following services are currently exempt from them:

When will this be enforced by D365FO?

The new user-based service protection API limits will be available to enable in Finance and Operations apps environments with version 10.0.28. In version 10.0.29, with 2022 release wave 2, the API limits will be enabled by default in all environments. The limits will be optional, allowing environment administrators to temporarily disable them should additional time be needed to optimize integrations. In version 10.0.32, with 2023 release wave 1, the API limits will be mandatory and the option to disable API limits will no longer be provided.

D365 FO user-based service protection API Time line

What does this technically mean for the D365FO integrations?

Each consumer of the OData service/custom service can potentially receive an HTTP 429 status code (too many requests). This response also contains a calculated “Retry-After” value, which can be used to retry the request after a specific number of seconds (more information).

What should you do now?

  1. Identify the integrations volume and their peak volumes
    1. Identify the volumes of your individual integrations and the number of calls the integration is making to D365FO.
    1. Then identify their peak volumes and see whether it’s well within the limits.
    1. See whether the integration is sharing service account or service principal and if they are sharing the account, then see whether combined volume could cause throttling.
  2. Distribute workloads across multiple service principals

User-based service protection API limits are implemented on a per-user basis. If all your integration is using a single service principal to perform a large number of data operations with D365FO, then the service protection API limits can be reached fairly quickly. This situation can be prevented using the service principle per integration. In the case of batch bulk operations, we can prevent this situation by distributing the workload in smaller batches across multiple service principals.

  • Implement the Retry mechanism based on the Retry-After response value from D365FO

Let the server tell you when to retry the request and don’t try to calculate the number of requests that you should send at a time.  Depend on the Retry-After interval value of the service protection API limit to tell you when to send more. That value will help keep your total throughput at the highest possible level.

  • Remove the affinity cookie

When you make a connection to a service in Azure, a cookie is returned that has the response. All your subsequent requests will try to be routed to the same server, unless capacity management forces them to go to another server. If you remove the cookie, every request that you send will be routed to any of the eligible servers. Therefore, throughput increases because limits are applied per server. This strategy lets you use more servers if they are available.

More Info :
MSFT DOCS
MSFT Presentation

Published by Poojith Jain

Poojith Jain is an Azure Architect with good experience with software design and development. He has a thorough knowledge of Azure Integration and he is passionate about solving complex and challenging problems in the field of Azure

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: