Tagged: azure

Cannot start analytics Tracker Exception – The Certificate was not found – Sitecore Azure Webapps

ERROR Cannot start analytics Tracker Exception: System.InvalidOperationException Message: The certificate was not found. Store: My, Location: CurrentUser, FindType: FindByThumbprint, FindValue: 23ACB78F3CDA99BA00646EA867C77466EBE8C718, InvalidAllowed: False. Source: Sitecore.Xdb.Common.Web at Sitecore.Xdb.Common.Web.Synchronous.SynchronousExtensions.SuspendContextLock[TResult](Func`1 taskFactory) at Sitecore.Analytics.DataAccess.Dictionaries.DataStorage.ReferenceDataClientDictionary.EnsureDefinitionType(String definitionTypeName) at Sitecore.Analytics.DataAccess.Dictionaries.DataStorage.ReferenceDataClientDictionary.LoadAs[T](Object key) at Sitecore.Analytics.DataAccess.Dictionaries.AverageCounterExtensions.MeasureMilliseconds[T](AverageCounter counter, Func`1 func) at Sitecore.Analytics.DataAccess.Dictionaries.ReferenceDataDictionary`2.Get(TKey key, LookupStrategy strategy) at Sitecore.Analytics.DataAccess.Dictionaries.UserAgentsDictionary.Register(String userAgentName) at Sitecore.Analytics.Tracking.CurrentVisitContext.set_UserAgent(String value) at Sitecore.Analytics.Pipelines.CreateVisits.InitializeWithRequestData.Process(CreateVisitArgs args) at (Object , Object ) at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args) at Sitecore.Pipelines.DefaultCorePipelineManager.Run(String pipelineName, PipelineArgs args, String pipelineDomain, Boolean failIfNotExists) at Sitecore.Pipelines.DefaultCorePipelineManager.Run(String pipelineName, PipelineArgs args, String pipelineDomain) at Sitecore.Analytics.Pipelines.CreateVisits.CreateVisitPipeline.Run(CreateVisitArgs args) at Sitecore.Analytics.Tracking.StandardSession.CreateInteraction(HttpContextBase httpContext) at Sitecore.Analytics.Pipelines.InitializeTracker.CreateVisit.Process(InitializeTrackerArgs args) at (Object , Object ) at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args) at Sitecore.Pipelines.DefaultCorePipelineManager.Run(String pipelineName, PipelineArgs args, String pipelineDomain, Boolean failIfNotExists) at Sitecore.Pipelines.DefaultCorePipelineManager.Run(String pipelineName, PipelineArgs args, String pipelineDomain) at Sitecore.Analytics.Pipelines.InitializeTracker.InitializeTrackerPipeline.Run(InitializeTrackerArgs args) at (Object , Object ) at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args) at Sitecore.Pipelines.DefaultCorePipelineManager.Run(String pipelineName, PipelineArgs args, String pipelineDomain, Boolean failIfNotExists) at Sitecore.Pipelines.DefaultCorePipelineManager.Run(String pipelineName, PipelineArgs args, String pipelineDomain) at Sitecore.Analytics.Pipelines.StartTracking.StartTrackingPipeline.Run(StartTrackingArgs args) at Sitecore.Analytics.DefaultTracker.StartTracking()

 

This error came up after we upgrade the certificate or I would say after we removed the Expired Certificate and add the new valid Certificate on Webapp (Sitecore on Azure).

This causes analytics to stop working.

To solve this error, Certificate Thumbprint has to be updated at various locations:

  1. Configuration files:
    On different web-apps of scaled environments, this Thumbprint needs to be updated.1.1 In App_Config/ConnectionStrings.config file of the following web-apps one might need to update any/all of xconnect.collection.certificate, xdb.marketingautomation.operations.client.certificate, xdb.referencedata.client.certificate (if exist):
    – CM
    – CD (all CD web-apps)
    – MA-Ops
    – PRC (processing)

    1.2 One needs to update the value of key validateCertificateThumbprint in App_Config/AppSettings.config file in following WebApps.
    – XC-Collect
    – XC-RefData
    – XC- Search
    – MA-Rep
    – MA-Ops

    1.3 <CertificateThumbprint> Tag’s value of Config/production/Sitecore.IdentityServer.Host.xml in following WebApp:
    – Si (Sitecore Identity)Once these configuration files of various Web-Apps has been updated, restart these Web-Apps.

  2.  If the above doesn’t solve the issue, following Web-App configurations also needs to be changed like below. Open the Configuration tab, and check for “Certificate” – all the Thumbprint needs to be updated where it is referring to old/incorrect Thumbprint:2.1 CM:
    Open the CM web-app in Azure –>Go to Configuration Tab –> And edit the certificates’ configuration.
    Make sure you update the Thumbprint value to valid/correct Thumbprint.
    – Cortex Reporting Client Certificate
    – WEBSITE_LOAD_CERTIFICATES
    – XConnect Collection Certificate
    – XDB MA Ops Client Certificate
    – XDB MA Reporting Client Certificate
    – XDB Reference Data Client Certificate
    2.2 SI:
    Update the valid/correct Thumbprint for Certificate Configuration in Sitecore-Identity WebApp as well.
    Replace the expired/incorrect thumbprint for “Certificate Thumbprint” & “WEBSITE_LOAD_CERTIFCATES”

    2.3 CDs:
    Go to the configuration tab for each CD server and update the thumbprint value for the following configuration:
    – WEBSITE_LOAD_CERTIFICATES
    – XConnect Collection Certificate
    – XDB MA Ops Client Certificate
    – XDB Reference Data Client Certificate

    2.4 Cortex Processing:
    From the configuration node, update the following Certificate configuration with valid/correct Thumbprint for Cortex Processing Web-App:
    – Processing Engine Xconnect Collection Client Certificate Thumbprint
    – Processing Engine Xconnect Search Client Certificate Thumbprint
    – WEBSITE_LOAD_CERTIFICATES
    – XConnect Server Certificate Validation Thumbprint

    2.5 Cortext Reporting:
    Update the below two certificate configuration for Cortext Reporting:
    – WEBSITE_LOAD_CERTIFICATES
    – XConnect Server Certificate Validation Thumbprint

    2.6 Ma-Ops
    Update the WEBSITE_LOAD_CERTIFICATES thumbprint for Ma-ops

    2.7 Ma-Rep
    Update below two configurations with the correct/valid thumbprint value:
    – WEBSITE_LOAD_CERTIFICATES
    – XConnect Server Certificate Validation Thumbprint

    2.8 Prc (Processing)

    Update the below two Certificate configuration with appropriate certificate thumbprint value:
    – WEBSITE_LOAD_CERTIFICATES
    – XConnect Collection Certificate

    2.9 XC-Collect
    Update the configuration for XC-Collect as well:
    – XConnect Server Certificate Validation Thumbprint

    2.10 XC-RefData
    Update the “XConnect Server Certificate Validation Thumbprint” configuration for XC-RefData Web-App as well.

 

Once you update these configurations, restart these Web-Apps and then try again. The analytics tracker error due to the Certificate was no longer in your log files.

 

Happy Sitecoring…

Sitecore Media Library on Microsoft Azure Cloud Storage with language version

This blog guides to configure the AzureCDN Media Upload. Media files are uploaded to azure at the time of publishing and hence the workflow can also be attached to media files. Also, it works with Media having Language versions When user renders the site, media items are fetched from the Azure but when content editor renders the site in page editor it will fetch the media from Sitecore so that it can be edited easily.

So starting with the Azure account first,

  1. You need to create Azure account. Create a new blob storage and a container. Please choose the name appropriate. Storage Account Name and Primary Access Key is required for connection string.azure account setup
    You are now done with the Azure part.
  2. Now in Visual Studio, Install the NuGet package “WindowsAzure.Storage” in your project.WindowsAzure.StorageOnce the NugetPackage is installed, we are to jump into the code.
  3. Let’s do some config changes. In web.config, add following keys under <appSettings>
    <appSettings>
        <add key="StorageConnectionString" value="DefaultEndpointsProtocol=https;AccountName=AZURE-ACCOUNT-NAME;AccountKey=PRIMARY-ACCOUNT-KEY" />
        <add key="AzureContainer" value="AZURE-CONTAINER-NAME" />
        <add key="OrignalPrefix" value="AZURE-URL-WITH-CONTAINER-NAME" />
    

    OrignalPrefix contains the value like https://testAccount.blob.core.windows.net/ContainerName.

  4. Let’s start with the AzureStorageUpload Code.AzureStorageUploadCS
    Create a class AzureStorageUpload.cs. This class contains the logic to upload/update/delete media to/from Azure Storage.

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Web;
    using Microsoft.WindowsAzure;
    using Microsoft.WindowsAzure.Storage;
    using Microsoft.WindowsAzure.Storage.Auth;
    using Microsoft.WindowsAzure.Storage.Blob;
    using System.Configuration;
    using Sitecore.Data.Items;
    using System.Diagnostics;
    using Sitecore.Diagnostics;
    
    namespace Website.Custom
    {
        public class AzureStorageUpload
        {
            CloudStorageAccount storageAccount;
            CloudBlobClient blobClient;
            CloudBlobContainer container;
            public AzureStorageUpload()
            {
                //use ConfigurationManager to retrieve the connection string
                 storageAccount = CloudStorageAccount.Parse(ConfigurationManager.AppSettings["StorageConnectionString"].ToString());
    
                //create a CloudBlobClient object using the storage account to retrieve objects that represent containers and blobs stored within the Blob Storage Service
                 blobClient = storageAccount.CreateCloudBlobClient();
    
                // Retrieve a reference to a container.
                 container = blobClient.GetContainerReference(ConfigurationManager.AppSettings["AzureContainer"]);
    
                // Create the container if it doesn't already exist.
                container.CreateIfNotExists();
    
                //By default, the new container is private and you must specify your storage access key to download blobs from this container. If you want to make the files within the container available to everyone, you can set the container to be public
                container.SetPermissions(
                    new BlobContainerPermissions
                    {
                        PublicAccess = BlobContainerPublicAccessType.Blob
                    });
            }
           public void uploadMediaToAzure(MediaItem mediaItem, string extension = "", string language = "")
            {
              
                // Retrieve reference to a blob.            
                CloudBlockBlob blockBlob = container.GetBlockBlobReference(mediaItem.MediaPath.TrimStart('/').Replace(mediaItem.DisplayName, mediaItem.ID.ToString().Replace("{", "").Replace("}", "").Replace("-", "")) + "-" + language + "." + extension);
                blockBlob.DeleteIfExists();
                if (string.IsNullOrEmpty(mediaItem.Extension))
                    return;
                // Create or overwrite the "myblob" blob with contents from a local file.
                if (mediaItem.HasMediaStream("Media"))
                {
                    using (var fileStream = (System.IO.FileStream)mediaItem.GetMediaStream())
                    {
                        blockBlob.UploadFromStream(fileStream);
                    }
                }
                else
                {
                    blockBlob.DeleteIfExists();
                }            
            }
    
    
    
            public void deleteMediaFromAzure(MediaItem mediaItem, string extension = "", string language = "")
            {   
                //Get the reference of the blob and delete it if exist.
                CloudBlockBlob blockBlob = container.GetBlockBlobReference(mediaItem.MediaPath.TrimStart('/').Replace(mediaItem.DisplayName, mediaItem.ID.ToString().Replace("{", "").Replace("}", "").Replace("-", "")) + "-" + language + "." + extension);
                blockBlob.DeleteIfExists();
            }
    
            public void replaceMediaFromAzure(MediaItem mediaItem, string extension="", string language = "")
            {         
                CloudBlockBlob blockBlob = container.GetBlockBlobReference(mediaItem.MediaPath.TrimStart('/').Replace(mediaItem.DisplayName, mediaItem.ID.ToString().Replace("{", "").Replace("}", "").Replace("-", "")) + "-" + language + "." + extension);
                blockBlob.DeleteIfExists();
                if (string.IsNullOrEmpty(mediaItem.Extension))
                    return;
                using (var fileStream = (System.IO.FileStream)mediaItem.GetMediaStream())
                {
                    blockBlob.UploadFromStream(fileStream);
                }
                
            }
        }
    }
    
  5. Now we are going to write the code of Publishing, What will do is, when the item get published, we will upload the media to AZURE account. And at the end, we will use this class as a processor and patch it in pipeline.
    using Sitecore;
    using Sitecore.Data.Items;
    using Sitecore.Diagnostics;
    using Sitecore.Globalization;
    using Sitecore.Jobs;
    using Sitecore.Publishing;
    using Sitecore.Publishing.Pipelines.PublishItem;
    using System;
    using System.Collections.Generic;
    using System.IO;
    using System.Linq;
    using System.Threading.Tasks;
    using System.Web;
    
    namespace Website.Custom
    {
        public class CdnPublishing : PublishItemProcessor
        {
            public string Enabled { get; set; }
            
            public override void Process(PublishItemContext context)
            {
                //Check the configuration to run the processor or not
                if (this.Enabled.ToLower() != "yes")
                    return;
                Log.Debug("Performing CDN validations", (object)this);
                Assert.ArgumentNotNull((object)context, "context");
                //Get the Context Item
                Item sourceItem = context.PublishHelper.GetSourceItem(context.ItemId);
                //If the source item is null, get the target item (specifically used for deleted item)
                if (sourceItem == null || !sourceItem.Paths.IsMediaItem)
                {
                    Item webSourceItem = context.PublishHelper.GetTargetItem(context.ItemId);
                    if (webSourceItem == null || !webSourceItem.Paths.IsMediaItem)
                    {
                        return;
                    }
                    else
                    {
                        sourceItem = webSourceItem;
                    }
    
                }
                MediaItem mediaItem = (MediaItem)sourceItem;
                string mediaExtension = mediaItem.Extension;
              //Get the Media Stream
                Stream mediaStream = mediaItem.GetMediaStream();            
                if (mediaStream == null || mediaStream.Length == 0L)
                {
                    if (((MediaItem)context.PublishHelper.GetTargetItem(context.ItemId)).GetMediaStream() == null || ((MediaItem)context.PublishHelper.GetTargetItem(context.ItemId)).GetMediaStream().Length == 0L)
                        return;
                    else
                        mediaExtension = ((MediaItem)context.PublishHelper.GetTargetItem(context.ItemId)).Extension;                    
                }
                AzureStorageUpload azureStorageUpload = new AzureStorageUpload();
                Log.Debug("Starting CDN synchonization", (object)this);
                try
                {
                    //Get Version Information
                    Item versionToPublish = context.VersionToPublish;
                    if (versionToPublish == null)
                    {
                        if (context.PublishHelper.GetTargetItemInLanguage(context.ItemId, sourceItem.Language) != null)
                            versionToPublish = context.PublishHelper.GetTargetItemInLanguage(context.ItemId, sourceItem.Language);
                    }
    
                    if (versionToPublish != null)
                    {
                        //Parameters to upload/replace/delete from on Azure
                        object[] args = new object[] { mediaItem, mediaExtension,versionToPublish.Language.Name };                    
                        Sitecore.Jobs.JobOptions jobOptions = null;
                        Context.Job.Status.State = JobState.Initializing;                    
                        if (context.Action == PublishAction.None)
                        {
    
                            jobOptions = new Sitecore.Jobs.JobOptions(
                                mediaItem.ID.ToString(),                     // identifies the job
                                "CDN Upload",                 // categoriezes jobs
                                Sitecore.Context.Site.Name,         // context site for job
                                azureStorageUpload,                  // object containing method
                                "uploadMediaToAzure",                  // method to invoke
                                args)                               // arguments to method
                                {
                                    AfterLife = TimeSpan.FromSeconds(5),  // keep job data for one hour
                                    EnableSecurity = false,             // run without a security context
                                };
                            Context.Job.Status.State = JobState.Finished;
                            Sitecore.Jobs.Job pub = Sitecore.Jobs.JobManager.Start(jobOptions);                        
                        }
                        if (context.Action == PublishAction.PublishSharedFields || context.Action == PublishAction.PublishVersion)
                        {
                            jobOptions = new Sitecore.Jobs.JobOptions(mediaItem.ID.ToString(), "CDN Upload", Sitecore.Context.Site.Name, azureStorageUpload, "replaceMediaFromAzure", args) { AfterLife = TimeSpan.FromSeconds(5), EnableSecurity = false, };                        
                            Context.Job.Status.State = JobState.Finished;
                            Sitecore.Jobs.Job pub = Sitecore.Jobs.JobManager.Start(jobOptions);                                          
                        }
                        //If the publish action is delete target item, get all the language versions of the item and delete it from Azure
                        if (context.Action == PublishAction.DeleteTargetItem)
                        {
                            foreach(Language lang in context.PublishOptions.TargetDatabase.GetLanguages())
                            {
                                mediaItem = context.PublishHelper.GetTargetItemInLanguage(mediaItem.ID, lang);
                                args = new object[] { mediaItem, mediaItem.Extension, lang.Name };
                                jobOptions = new Sitecore.Jobs.JobOptions(mediaItem.ID.ToString(), "CDN Upload", Sitecore.Context.Site.Name, azureStorageUpload, "deleteMediaFromAzure", args)
                                {
                                    AfterLife = TimeSpan.FromSeconds(5),
                                    EnableSecurity = false,
                                };
                                Context.Job.Status.State = JobState.Finished;
                                Sitecore.Jobs.Job pub = Sitecore.Jobs.JobManager.Start(jobOptions);
                            }                        
                        }                 
                    }
                }             
                catch (Exception ex)
                {
                    Exception exception = new Exception(string.Format("CDN Processing failed for {1} ({0} version: {2}). {3}", (object)sourceItem.ID, (object)sourceItem.Name, (object)context.VersionToPublish.Language.Name, (object)ex.Message));
                    Log.Error(exception.Message, exception, (object)this);
                    context.Job.Status.Failed = true;
                    context.Job.Status.Messages.Add(exception.Message);
                }
                Log.Debug(" CDN synchronization finished ", (object)this);
            }
        }
    }
    

    Now let’s patch this class in pipeline.

  6. Instead of changing the configuration, we can patch the processor from a different file to keep the web.config intact.Create a .config file under App_Config\Include\AzurePublishing.config:
    <configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
      <sitecore>
        <pipelines>
          <publishItem>
              <processor type="Website.Custom.CdnPublishing,Website"
            patch:before="processor[@type='Sitecore.Publishing.Pipelines.PublishItem.PerformAction, Sitecore.Kernel']">          
                <!--If yes, this custom CDN publishing processor will be executed otherwise not. Values: yes|no-->
              <Enabled>yes</Enabled>            
            </processor>
          </publishItem>
        </pipelines>
      </sitecore>
    </configuration>
    
  7. Now, to fetch the media from Azure, we need to make changes in Media Provider. But also we need to make sure that for Page Editor and Preview , the media must be fetched from Sitecore only. And to render the site, media must be fetched from Azure. So here we go,
    using Sitecore.Data.Items;
    using Sitecore.Events.Hooks;
    using Sitecore.Resources.Media;
    using System;
    using System.Collections.Generic;
    using System.IO;
    using System.Linq;
    using System.Text;
    using System.Web;
    
    namespace Website.Custom
    {
        public class MediaProvider : Sitecore.Resources.Media.MediaProvider, IHook
        {
            public void Initialize()
            {
                MediaManager.Provider = this;
            }
    
            public override string GetMediaUrl(MediaItem item)
            {
                string mediaUrl = base.GetMediaUrl(item);
                return mediaUrl;
            }
    
            public override string GetMediaUrl(MediaItem item, MediaUrlOptions options)
            {            
                string mediaUrl = base.GetMediaUrl(item, options);            
                return GetCstmMediaUrl(mediaUrl, item);
            }
    
            /// <summary>
            /// Determines if media should be pulling from the CDN or not
            /// </summary>
            /// <param name="mediaUrl"></param>
            /// <param name="item"></param>
            /// <returns></returns>               
            public string GetCstmMediaUrl(string mediaUrl, MediaItem item)
            {
                //verify the domain was set in the config
                if (string.IsNullOrEmpty(OriginPrefix))
                {
                    return mediaUrl;
                }
    
                //Condition to fetch the Azure url only if the site is rendered and not for Preview or editing mode.            
                if(Sitecore.Context.GetSiteName().ToLower() != "website" || Sitecore.Context.PageMode.IsPageEditor||Sitecore.Context.PageMode.IsPreview||Sitecore.Context.PageMode.IsSimulatedDevicePreviewing)
                {
                    return mediaUrl;
                }
                mediaUrl = mediaUrl.Replace("~/media/", "/");            
                //this happens while indexing unless the proper site is set
                mediaUrl = mediaUrl.Replace("/sitecore/shell/", "/");
                mediaUrl = mediaUrl.Replace("//", "/");
                //reference the file in the cdn by the actual extension
                mediaUrl = mediaUrl.Replace(".ashx", "." + item.Extension);
                mediaUrl = string.Format("{0}{1}", OriginPrefix,mediaUrl);           
                string Language = string.Empty;
                if (mediaUrl.Contains("la="))
                {
                    Language = mediaUrl.Substring(mediaUrl.IndexOf("la=") + 3);
                    if (Language.Contains("&"))
                        Language = Language.Substring(0, Language.IndexOf("&"));
                }
                else
                {
                    Language = Sitecore.Configuration.Settings.DefaultLanguage;
                }
                //create the media url accoriding to the naming convention of Azure-media-file name
                mediaUrl= mediaUrl.Replace(item.DisplayName + "." + item.Extension, item.ID.ToString().Replace("{", "").Replace("}", "").Replace("-", "")+"-"+Language + "." + item.Extension);            
                if (HttpContext.Current != null && HttpContext.Current.Request.IsSecureConnection)
                {
                    //if we are on a secure connection, make sure we are making an https url over to the cdn
                    mediaUrl = mediaUrl.Replace("http://", "https://");
                }
               return mediaUrl;
            }
    
            
            public string OriginPrefix
            {
                get
                {
                    return System.Configuration.ConfigurationManager.AppSettings["OrignalPrefix"];
                    
                }
                //set;
            }
    
          
        }
    }
    
  8. Now we need  to replace the Sitecore default MediaProvider with our custom Media Provider.
    In web.config, replace the default MediaProvider with this one.

    <!--<mediaProvider type="Sitecore.Resources.Media.MediaProvider, Sitecore.Kernel" />-->
    <mediaProvider type="Website.Custom.MediaProvider, Website" />
    

 

So, that’s it..  Publish any media item, and you will find it in your Azure storage account.

Things to notice here in Publishing processor , we have used Jobs to publish the items. Reason is, uploading a media synchronously on Azure takes time, and if there are around 1000 + items it will take long time and using Jobs – we can reduce it into seconds.