Fixing an XM Cloud Deployment Failure / Blogs / Perficient

Intro ????

Last week, I noticed that deployments to Sitecore XM Cloud were failing on one of my projects. In this blog post, I’ll review the troubleshooting steps I went through and what the issue turned out to be. To provide a bit more context on the DevOps setup for this particular project, an Azure DevOps pipeline runs a script. That script uses the Sitecore CLI and the Sitecore XM Cloud plugin’s cloud deployment command to deploy to XM Cloud. The last successful deployment was just a few days prior and there hadn’t been many code changes since. Initially, I was pretty stumped but, hey, what can you do except start from the top…

Troubleshooting ????‍♂️

1. Anyone that has worked with cloud-based SaaS services knows that transient faults are a thing–and XM Cloud is no exception. The first thing I tried was to simply rerun the failed stage in our pipeline to see if this was “just a hiccup.” Alas, several subsequent deployment attempts failed with the same error. Okay, fine, this wasn’t a transient issue ????.
2. Looking at the logs in the XM Cloud Deploy interface, the build stage was consistently failing. Drilling into the logs, there were several compilation errors citing missing Sitecore assemblies. For example: error CS0246: The type or namespace name ‘Item’ could not be found (are you missing a using directive or an assembly reference?). This suggested an issue with either the NuGet restore or with compilation more broadly.
3. Rerunning failed stages in an Azure DevOps pipeline uses the same commit that was associated with the first run–the latest code from the branch isn’t pulled on each rerun attempt. This meant that the code used for the last successful deployment was the same code used for the subsequent attempts. In other words, this probably wasn’t a code issue (famous last words, right ?????).
4. Just to be sure, I diffed several recent commits on our development branch and, yeah, there weren’t any changes that could have broken compilation since the last successful deployment.
5. To continue the sanity checks, I pulled down the specific commit locally and verified that I could:
   1. Restore NuGet packages, via both the UI and console
   2. Build/rebuild the Visual Studio solution
6. After revisiting and diffing the XM Cloud Deploy logs, I noticed that the version of msbuild had changed between the last successful deployment and the more recent failed deployments. I downloaded the same, newer version of msbuild and verified, once again, that I could restore NuGet packages and build the solution.
7. Finally, I confirmed that the validation build configured for the development branch (via branch policies in Azure DevOps) was running as expected and successfully building the solution each time a new pull request was created.

At this point, while I continued to analyze the deployment logs, I opened a Sitecore support ticket to have them weigh in ????‍♂️. I provided support with the last known working build logs, the latest failed build logs, and the list of my troubleshooting steps up to that point.

The Fix ????

After hearing back from Sitecore support, it turned out that Sitecore had recently made a change to how the buildTargets property in the xmcloud.build.json file was consumed and used as part of deployments. To quote the support engineer:

There were some changes in the build process, and now the build targets are loaded from the “buildTargets ” list. The previous working builds were using the “.sln” file directly.
It looks like that resulted in the build not working properly for some projects.

The suggested fix was to specifically target the Visual Studio solution file to ensure that the XM Cloud Deployment NuGet package restore and compilation worked as expected. My interpretation of the change was “XM Cloud Deploy used to not care about/respect buildTargets…but now it does.”

After creating a pull request to change the buildTargets property from this (targeting the specific, top-level project):

{
  ...
  "buildTargets": [
    "./src/platform/Project/Foo.Project.Platform/Platform.csproj"
  ]
  ...
}

…to this (targeting the solution):

{
  ...
  "buildTargets": [
    "./Foo.sln"
  ]
  ...
}

…the next deployment to XM Cloud (via CI/CD) worked as expected. ✅????

After asking the Sitecore support engineer where this change was documented, they graciously escalated internally and posted a new event to the Sitecore Status Page to acknowledge the change/issue: Deployment is failing to build.

If you’re noticing that your XM Cloud deployments are failing on the build step while compiling your Visual Studio solution, make sure you’re targeting the solution file (.sln) and not a specific project file (.csproj) in the buildTargets property in the xmcloud.build.json file…because it matters now, apparently ????.

Thanks for the read! ????