Fixing timeout exception on CSOM based site collection creation

  • [アーティクル]

One common question or challenge what we see community to face is the time out exception when site collection creation code is executed or when you apply needed customizations to just created SharePoint site collection. Since this is relatively common question in Yammer groups or in blog posts, thought that it would be worth while to explain the challenge and how to avoid issues in your provisioning solution. I’m partly writing this from selfish reasons, so that I can simply refer to this blog post when the question is again asked.

Most common reason for this exception is that the host process running the code is actually timing out, which can happen in the Office 365 or in on-premises. You can however avoid this situation with properly written code or adjusting environmental variables, slightly depending on the used environment. Root cause for the exception is commonly that process executing the request in the IIS web site in on-premises or Azure web site will simply time out.

Here’s logical drawing on what’s actually happening step-by-step related on the timeout issues with IIS and Azure web sites.

image

  1. User is moving to the context of the provider hosted add-in/app for performing actions
  2. User is performing needed operations and starts process in the provider hosted add-in/app side
  3. Process is performing remote API commands towards SharePoint and/or any other service using remote APIs
  4. If the start of the your server side execution takes too long, process or connection will be shut down automatically. Timeout for Azure web sites is 60 seconds and in on-premises this value slightly depends on server version, but it’s not good practice to execute long lasting code in the web site process due numerous reasons.

Fixing this issue is dependent on your hosting environment, so let’s have a look on the right design depending on the used environment.

How to fix the issue in Azure web sites

Executions in Azure web sites will timeout in 60 seconds, like mentioned above. This means that if your code execution will take longer than 60 seconds, your process will time out and the execution will return to the browser. All commands executed before the timeout exception will be completed asynchronously, but any code after the last executed CSOM operation will not be completed. Typically you’d see for example site collection to be eventually created, but any code applying needed customizations to the just created site collection would not be executed.

Key difference for the SharePoint Online based site collection creation, compared to on-premises, is that the Tenant.CreateSite method is asynchronous, so if you’d like to apply any customizations on top of the just created site collection, you’ll have to wait for the site collection creation to complete by checking the creation status in while loop. This can be done by checking the IsComplete property in while loop like in the following code take from Provisioning.Cloud.Async.WebJob sample.

 var tenant = new Tenant(adminCtx);
 var properties = new SiteCreationProperties()
 {
     Url = webFullUrl,
     Owner = provisionData.RequestData.Owner,
     Title = provisionData.RequestData.Title,
     Template = provisionData.RequestData.Template,
     TimeZoneId = provisionData.RequestData.TimeZoneId,
     Lcid = provisionData.RequestData.Lcid,
     StorageMaximumLevel = provisionData.RequestData.StorageMaximumLevel
 };
  
 //start the SPO operation to create the site
 SpoOperation op = tenant.CreateSite(properties);
 adminCtx.Load(tenant);
 adminCtx.Load(op, i => i.IsComplete);
 adminCtx.ExecuteQuery();
  
 //check if site creation operation is complete
 while (!op.IsComplete)
 {
     //wait 15 seconds and try again
     System.Threading.Thread.Sleep(15000);
     op.RefreshLoad();
     adminCtx.ExecuteQuery();
 }
  
 // Apply branding if theme information is provided
 if (!string.IsNullOrEmpty(provisionData.BrandingData.ThemeName))
 {
     ApplyTemplateForCreatedSiteCollection(webFullUrl, provisionData);
 }

So how to fix timeout challenge for Azure web sites?

You cannot change the timeout setting for the Azure web sites, so you should be looking into provisioning site collections using asynchronous pattern. This means that your web site code will only add a task to be completed for background process, which will pickup the requests and process them one-by-one. Typically you’d use Azure WebJobs for completing this for example by adding request for the site collection creation to Azure storage queue which will automatically be processed by continuously running WebJob.

Here’s the logical design for the Azure implementation for processing site collection requests asynchronously.

image

  1. Provider hosted add-in/app is being provided for the end users to request site collections using self service model
  2. Provider hosted add-in/app has custom UI from where the end users can select wanted template
  3. Request to create new site collection is added to Azure storage queue for processing. Notice that you can also add additional processes for the request handling, like needed workflow approvals and other actions
  4. Continuous running Azure WebJob is instantiated automatically when message is added to Azure storage queue and processing of the request will start automatically. Actual site collection creation is performed using CSOM, including needed customizations like branding configuration.
  5. Actual site collection is created asynchronously and notifications are send for the requestor for

Notice. SharePoint Online had also recently (autumn 2015) some timeout challenges around the execution of the CreateSite method regardless of calling that from the background process. Fix for this issue has been completed and pushed to all Office 365 tenants world wide.

Reference solutions

Here’s few reference solutions around the right way to implement the site collection creation in the SharePoint online / Office 365. There’s few different samples with different complexity levels for ensuring that you can easily adapt the model.

 

How to fix the issue in on-premises

In on-premises you'd be using normal IIS to host your provider hosted add-in/app and even though you could adjust the IIS timeout from default settings, you should also look into using asynchronous model, so that end user’s browser session is not “high jacked” for the site collection creation. There’s also one really big difference on the CSOM implementation style between on-premises and SharePoint online. This is the fact that Tenant.CreateSite method is actually synchronous. This means that site collection is created synchronously when you call this method and you do not need to check the status of the creation in while loop, so the code for on-premises is looking something like following taken from Provisioning.OnPrem.Async sample.

 // Set the time out as high as possible
 actx.RequestTimeout = Timeout.Infinite;
  
 // Create tenant object - used for on-premises as well
 var tenant = new Tenant(actx);
  
 // Get the request details for the site creation object
 var properties = new SiteCreationProperties()
 {
     Url = webUrl,
     Owner = listItem["AdminAccount"].ToString(),
     Title = listItem["Title"].ToString(),
     Template = listItem["Template"].ToString(),
 };
  
 // Start the operation to create the site
 SpoOperation op = tenant.CreateSite(properties);
  
 // Notice that following line is synchronious in on-premises               
 actx.ExecuteQuery();
  
 // Apply needed customizations to just created site collection
 SetThemeToNewSite(webUrl);

So how fix the timeout challenge for on-premises?

First step is to use the asynchronous pattern in the on-premises as well, like mentioned, so that IIS web site execution timeout won’t cause issues. This could mean that you are for example storing site collection creation requests to a custom list at the on-premises SharePoint and then you’d have a remote timer job, which could be as simple as console application, which has been scheduled using windows scheduler, which accesses the list and processes requests as they arrive. Executing CSOM code from the console application does not have any specific timeouts, so from CSOM caller side you do not have any challenges, but server side is then the challenge.

Since in on-premises the creation of site collection is synchronous and it could take relatively long time to complete, you’d have to adjust the timeout of the CSOM calls in the SharePoint farm. This can be done by adjusting ExecutionTimeout property from the SPWebApplication.ClientCallableSettings object. Here’s a reference PowerShell to adjust this setting to 5 minutes. Notice that actual time required for site collection creation is highly dependent on your farm infrastructure, so you might have to adjust this setting depending on your environment.

 #
 # By default time out setting is 90 seconds, which might not be enough
 # for site collection creation. This setting can be controlled from the  
 # SPWebApplication.ClientCallableSettings. 
 #
 # This script will increase the setting to 5 minutes to ensure that
 # site collection creation is successful. 
 #
 $webAppUrl = "https://dev.contoso.com"
 $timeoutInMinutes = 5;
  
 $snapin = Get-PSSnapin | Where-Object {$_.Name -eq 'Microsoft.SharePoint.Powershell'}
 if ($snapin -eq $null) 
 {
     Write-Host "Loading SharePoint Powershell Snapin"
     Add-PSSnapin "Microsoft.SharePoint.Powershell"
 }
  
 # Get web application
 wa = Get-SPWebApplication -Identity $webAppUrl
 # Increase time out for CSOM calls - by default this is 90 seconds
 wa.ClientCallableSettings.ExecutionTimeout = [System.Timespan]::FromMinutes($timeoutInMinutes)
 wa.Update();
  
 # Output current setting
 wa = Get-SPWebApplication -Identity $webAppUrl
 wa.ClientCallableSettings
  

Reference solution

Here’s a reference solution for the on-premises implementation. This sample does not use the PnP Provisioning Engine, but it’s also supported for on-premises, so you could easily adapt the models from the cloud samples for on-premises as well.

  • Provisioning.OnePrem.Async – Similar asynchronous implementation as for the SharePoint online, but does not use Azure WebJobs.

 

Office 365 Developer Patterns and Practices

Office365PnPLogoRed_thumb1Techniques showed in this blog post are part of the Office 365 Developer Patterns and Practices guidance, which contains more than 100 samples and solutions demonstrating different patterns and practices related on the add-in/app model development together with additional documentation related on the app model techniques.

Check the details around PnP from dev.office.com at https://aka.ms/OfficeDevPnP. Please join us on sharing patterns and practices for the community for the benefit of the community. If you have any questions, comments or feedback related on this sample, blog post or anything on PnP, please use the Office 365 Developer Patterns and Practices Yammer group at https://aka.ms/OfficeDevPnPYammer.

“Sharing is caring”

Comments

  • Anonymous
    December 06, 2015
    I had not read this article carefully, but i am using a different approach for SharePoint on-premise, the basic idea is that we associate SharePoint object such as Site with SharePoint managed metadata Terms. For Site Creation, it can be done in two phase, first, user fill in necessary information and the information is committed in a term. Next, a work item timer job can create the Site with the information in a Term. If failed, the timer job can run again, user do not have to input the information again, since usually, commit to term won't fail. social.msdn.microsoft.com/.../creating-a-factory-class-with-sharepoint-server-side-api-to-instantiate-a-wrapper-class-around

  • Anonymous
    December 08, 2015
    Hi GuYuming, That's absolutely valid solution. In your case you have just decided to store the requests to term store and not for example to standard SharePoint list, but the logical design is exactly the same with disconnected handling of the request and then having a separate remote timer job processing them. Thx for sharing this alternative approach as well.