Tagged: azure storage upload
Sitecore Media Library on Microsoft Azure Cloud Storage with language version
This blog guides to configure the AzureCDN Media Upload. Media files are uploaded to azure at the time of publishing and hence the workflow can also be attached to media files. Also, it works with Media having Language versions When user renders the site, media items are fetched from the Azure but when content editor renders the site in page editor it will fetch the media from Sitecore so that it can be edited easily.
So starting with the Azure account first,
- You need to create Azure account. Create a new blob storage and a container. Please choose the name appropriate. Storage Account Name and Primary Access Key is required for connection string.
You are now done with the Azure part. - Now in Visual Studio, Install the NuGet package “WindowsAzure.Storage” in your project.Once the NugetPackage is installed, we are to jump into the code.
- Let’s do some config changes. In web.config, add following keys under <appSettings>
<appSettings> <add key="StorageConnectionString" value="DefaultEndpointsProtocol=https;AccountName=AZURE-ACCOUNT-NAME;AccountKey=PRIMARY-ACCOUNT-KEY" /> <add key="AzureContainer" value="AZURE-CONTAINER-NAME" /> <add key="OrignalPrefix" value="AZURE-URL-WITH-CONTAINER-NAME" />
OrignalPrefix contains the value like https://testAccount.blob.core.windows.net/ContainerName.
- Let’s start with the AzureStorageUpload Code.
Create a class AzureStorageUpload.cs. This class contains the logic to upload/update/delete media to/from Azure Storage.using System; using System.Collections.Generic; using System.Linq; using System.Web; using Microsoft.WindowsAzure; using Microsoft.WindowsAzure.Storage; using Microsoft.WindowsAzure.Storage.Auth; using Microsoft.WindowsAzure.Storage.Blob; using System.Configuration; using Sitecore.Data.Items; using System.Diagnostics; using Sitecore.Diagnostics; namespace Website.Custom { public class AzureStorageUpload { CloudStorageAccount storageAccount; CloudBlobClient blobClient; CloudBlobContainer container; public AzureStorageUpload() { //use ConfigurationManager to retrieve the connection string storageAccount = CloudStorageAccount.Parse(ConfigurationManager.AppSettings["StorageConnectionString"].ToString()); //create a CloudBlobClient object using the storage account to retrieve objects that represent containers and blobs stored within the Blob Storage Service blobClient = storageAccount.CreateCloudBlobClient(); // Retrieve a reference to a container. container = blobClient.GetContainerReference(ConfigurationManager.AppSettings["AzureContainer"]); // Create the container if it doesn't already exist. container.CreateIfNotExists(); //By default, the new container is private and you must specify your storage access key to download blobs from this container. If you want to make the files within the container available to everyone, you can set the container to be public container.SetPermissions( new BlobContainerPermissions { PublicAccess = BlobContainerPublicAccessType.Blob }); } public void uploadMediaToAzure(MediaItem mediaItem, string extension = "", string language = "") { // Retrieve reference to a blob. CloudBlockBlob blockBlob = container.GetBlockBlobReference(mediaItem.MediaPath.TrimStart('/').Replace(mediaItem.DisplayName, mediaItem.ID.ToString().Replace("{", "").Replace("}", "").Replace("-", "")) + "-" + language + "." + extension); blockBlob.DeleteIfExists(); if (string.IsNullOrEmpty(mediaItem.Extension)) return; // Create or overwrite the "myblob" blob with contents from a local file. if (mediaItem.HasMediaStream("Media")) { using (var fileStream = (System.IO.FileStream)mediaItem.GetMediaStream()) { blockBlob.UploadFromStream(fileStream); } } else { blockBlob.DeleteIfExists(); } } public void deleteMediaFromAzure(MediaItem mediaItem, string extension = "", string language = "") { //Get the reference of the blob and delete it if exist. CloudBlockBlob blockBlob = container.GetBlockBlobReference(mediaItem.MediaPath.TrimStart('/').Replace(mediaItem.DisplayName, mediaItem.ID.ToString().Replace("{", "").Replace("}", "").Replace("-", "")) + "-" + language + "." + extension); blockBlob.DeleteIfExists(); } public void replaceMediaFromAzure(MediaItem mediaItem, string extension="", string language = "") { CloudBlockBlob blockBlob = container.GetBlockBlobReference(mediaItem.MediaPath.TrimStart('/').Replace(mediaItem.DisplayName, mediaItem.ID.ToString().Replace("{", "").Replace("}", "").Replace("-", "")) + "-" + language + "." + extension); blockBlob.DeleteIfExists(); if (string.IsNullOrEmpty(mediaItem.Extension)) return; using (var fileStream = (System.IO.FileStream)mediaItem.GetMediaStream()) { blockBlob.UploadFromStream(fileStream); } } } }
- Now we are going to write the code of Publishing, What will do is, when the item get published, we will upload the media to AZURE account. And at the end, we will use this class as a processor and patch it in pipeline.
using Sitecore; using Sitecore.Data.Items; using Sitecore.Diagnostics; using Sitecore.Globalization; using Sitecore.Jobs; using Sitecore.Publishing; using Sitecore.Publishing.Pipelines.PublishItem; using System; using System.Collections.Generic; using System.IO; using System.Linq; using System.Threading.Tasks; using System.Web; namespace Website.Custom { public class CdnPublishing : PublishItemProcessor { public string Enabled { get; set; } public override void Process(PublishItemContext context) { //Check the configuration to run the processor or not if (this.Enabled.ToLower() != "yes") return; Log.Debug("Performing CDN validations", (object)this); Assert.ArgumentNotNull((object)context, "context"); //Get the Context Item Item sourceItem = context.PublishHelper.GetSourceItem(context.ItemId); //If the source item is null, get the target item (specifically used for deleted item) if (sourceItem == null || !sourceItem.Paths.IsMediaItem) { Item webSourceItem = context.PublishHelper.GetTargetItem(context.ItemId); if (webSourceItem == null || !webSourceItem.Paths.IsMediaItem) { return; } else { sourceItem = webSourceItem; } } MediaItem mediaItem = (MediaItem)sourceItem; string mediaExtension = mediaItem.Extension; //Get the Media Stream Stream mediaStream = mediaItem.GetMediaStream(); if (mediaStream == null || mediaStream.Length == 0L) { if (((MediaItem)context.PublishHelper.GetTargetItem(context.ItemId)).GetMediaStream() == null || ((MediaItem)context.PublishHelper.GetTargetItem(context.ItemId)).GetMediaStream().Length == 0L) return; else mediaExtension = ((MediaItem)context.PublishHelper.GetTargetItem(context.ItemId)).Extension; } AzureStorageUpload azureStorageUpload = new AzureStorageUpload(); Log.Debug("Starting CDN synchonization", (object)this); try { //Get Version Information Item versionToPublish = context.VersionToPublish; if (versionToPublish == null) { if (context.PublishHelper.GetTargetItemInLanguage(context.ItemId, sourceItem.Language) != null) versionToPublish = context.PublishHelper.GetTargetItemInLanguage(context.ItemId, sourceItem.Language); } if (versionToPublish != null) { //Parameters to upload/replace/delete from on Azure object[] args = new object[] { mediaItem, mediaExtension,versionToPublish.Language.Name }; Sitecore.Jobs.JobOptions jobOptions = null; Context.Job.Status.State = JobState.Initializing; if (context.Action == PublishAction.None) { jobOptions = new Sitecore.Jobs.JobOptions( mediaItem.ID.ToString(), // identifies the job "CDN Upload", // categoriezes jobs Sitecore.Context.Site.Name, // context site for job azureStorageUpload, // object containing method "uploadMediaToAzure", // method to invoke args) // arguments to method { AfterLife = TimeSpan.FromSeconds(5), // keep job data for one hour EnableSecurity = false, // run without a security context }; Context.Job.Status.State = JobState.Finished; Sitecore.Jobs.Job pub = Sitecore.Jobs.JobManager.Start(jobOptions); } if (context.Action == PublishAction.PublishSharedFields || context.Action == PublishAction.PublishVersion) { jobOptions = new Sitecore.Jobs.JobOptions(mediaItem.ID.ToString(), "CDN Upload", Sitecore.Context.Site.Name, azureStorageUpload, "replaceMediaFromAzure", args) { AfterLife = TimeSpan.FromSeconds(5), EnableSecurity = false, }; Context.Job.Status.State = JobState.Finished; Sitecore.Jobs.Job pub = Sitecore.Jobs.JobManager.Start(jobOptions); } //If the publish action is delete target item, get all the language versions of the item and delete it from Azure if (context.Action == PublishAction.DeleteTargetItem) { foreach(Language lang in context.PublishOptions.TargetDatabase.GetLanguages()) { mediaItem = context.PublishHelper.GetTargetItemInLanguage(mediaItem.ID, lang); args = new object[] { mediaItem, mediaItem.Extension, lang.Name }; jobOptions = new Sitecore.Jobs.JobOptions(mediaItem.ID.ToString(), "CDN Upload", Sitecore.Context.Site.Name, azureStorageUpload, "deleteMediaFromAzure", args) { AfterLife = TimeSpan.FromSeconds(5), EnableSecurity = false, }; Context.Job.Status.State = JobState.Finished; Sitecore.Jobs.Job pub = Sitecore.Jobs.JobManager.Start(jobOptions); } } } } catch (Exception ex) { Exception exception = new Exception(string.Format("CDN Processing failed for {1} ({0} version: {2}). {3}", (object)sourceItem.ID, (object)sourceItem.Name, (object)context.VersionToPublish.Language.Name, (object)ex.Message)); Log.Error(exception.Message, exception, (object)this); context.Job.Status.Failed = true; context.Job.Status.Messages.Add(exception.Message); } Log.Debug(" CDN synchronization finished ", (object)this); } } }
Now let’s patch this class in pipeline.
- Instead of changing the configuration, we can patch the processor from a different file to keep the web.config intact.Create a .config file under App_Config\Include\AzurePublishing.config:
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/"> <sitecore> <pipelines> <publishItem> <processor type="Website.Custom.CdnPublishing,Website" patch:before="processor[@type='Sitecore.Publishing.Pipelines.PublishItem.PerformAction, Sitecore.Kernel']"> <!--If yes, this custom CDN publishing processor will be executed otherwise not. Values: yes|no--> <Enabled>yes</Enabled> </processor> </publishItem> </pipelines> </sitecore> </configuration>
- Now, to fetch the media from Azure, we need to make changes in Media Provider. But also we need to make sure that for Page Editor and Preview , the media must be fetched from Sitecore only. And to render the site, media must be fetched from Azure. So here we go,
using Sitecore.Data.Items; using Sitecore.Events.Hooks; using Sitecore.Resources.Media; using System; using System.Collections.Generic; using System.IO; using System.Linq; using System.Text; using System.Web; namespace Website.Custom { public class MediaProvider : Sitecore.Resources.Media.MediaProvider, IHook { public void Initialize() { MediaManager.Provider = this; } public override string GetMediaUrl(MediaItem item) { string mediaUrl = base.GetMediaUrl(item); return mediaUrl; } public override string GetMediaUrl(MediaItem item, MediaUrlOptions options) { string mediaUrl = base.GetMediaUrl(item, options); return GetCstmMediaUrl(mediaUrl, item); } /// <summary> /// Determines if media should be pulling from the CDN or not /// </summary> /// <param name="mediaUrl"></param> /// <param name="item"></param> /// <returns></returns> public string GetCstmMediaUrl(string mediaUrl, MediaItem item) { //verify the domain was set in the config if (string.IsNullOrEmpty(OriginPrefix)) { return mediaUrl; } //Condition to fetch the Azure url only if the site is rendered and not for Preview or editing mode. if(Sitecore.Context.GetSiteName().ToLower() != "website" || Sitecore.Context.PageMode.IsPageEditor||Sitecore.Context.PageMode.IsPreview||Sitecore.Context.PageMode.IsSimulatedDevicePreviewing) { return mediaUrl; } mediaUrl = mediaUrl.Replace("~/media/", "/"); //this happens while indexing unless the proper site is set mediaUrl = mediaUrl.Replace("/sitecore/shell/", "/"); mediaUrl = mediaUrl.Replace("//", "/"); //reference the file in the cdn by the actual extension mediaUrl = mediaUrl.Replace(".ashx", "." + item.Extension); mediaUrl = string.Format("{0}{1}", OriginPrefix,mediaUrl); string Language = string.Empty; if (mediaUrl.Contains("la=")) { Language = mediaUrl.Substring(mediaUrl.IndexOf("la=") + 3); if (Language.Contains("&")) Language = Language.Substring(0, Language.IndexOf("&")); } else { Language = Sitecore.Configuration.Settings.DefaultLanguage; } //create the media url accoriding to the naming convention of Azure-media-file name mediaUrl= mediaUrl.Replace(item.DisplayName + "." + item.Extension, item.ID.ToString().Replace("{", "").Replace("}", "").Replace("-", "")+"-"+Language + "." + item.Extension); if (HttpContext.Current != null && HttpContext.Current.Request.IsSecureConnection) { //if we are on a secure connection, make sure we are making an https url over to the cdn mediaUrl = mediaUrl.Replace("http://", "https://"); } return mediaUrl; } public string OriginPrefix { get { return System.Configuration.ConfigurationManager.AppSettings["OrignalPrefix"]; } //set; } } }
- Now we need to replace the Sitecore default MediaProvider with our custom Media Provider.
In web.config, replace the default MediaProvider with this one.<!--<mediaProvider type="Sitecore.Resources.Media.MediaProvider, Sitecore.Kernel" />--> <mediaProvider type="Website.Custom.MediaProvider, Website" />
So, that’s it.. Publish any media item, and you will find it in your Azure storage account.
Things to notice here in Publishing processor , we have used Jobs to publish the items. Reason is, uploading a media synchronously on Azure takes time, and if there are around 1000 + items it will take long time and using Jobs – we can reduce it into seconds.