By Joe Beernink on July 19, 2011

A coworker and I were in the process of migrating a client’s web server from old hardware to new hardware last week, and we needed to be able to test the system before turning it on to live traffic. The web applications called web services, and the configuration of the web app included the URL for the web services.

We needed to be careful what we were doing here, because the web services were also migrating to a new server, and on this new server, were pointing to a new database server as well. We didn’t want to change all the configuration settings on the new servers, and we couldn’t change the public DNS entry.

I’ve done this before, but not everyone knows about the following trick, so here it is:

On Windows, if you go into your %WinDir%\System32\drivers\etc directory, you’ll find a file called hosts, often referred to as “etc\hosts” or “Etsy Hosts”. Open this file with notepad running as administrator on your machine. The instructions for changing this file are right in there, and it’s pretty easy. We added a new line to file on the new server, (and on the test client) like:

127.0.0.1       www.palador.com

We saved our changes, and then restarted IIS on the box. And presto…

But it didn’t work.

We reset IIS again, and did an ipconfig /flushdns from the command prompt.

And it still pointed to the old domain.

We went back to the Hosts file and everything looked fine. I scratched my head. Then I realized that we had forgotten to put a carriage return after the new line. We added the hard-return, restarted IIS and flushed DNS, and now everything worked.

I’m pretty sure I had run into this exact same issue a few years ago, and that’s why I knew to look for it. Hopefully this will help someone else avoid this issue.

By Joe Beernink on June 02, 2011

During testing of an upcoming release of one of our Azure based projects, we began getting intermittent errors while uploading images (blobs) to Windows Azure storage using writable SharedAccessSignatures. I did a search on the web, and oddly enough, one of my previous posts covered a very similar intermittent 403 error. I went through my code and looked for places I might be using Uri.ToString instead of Uri.AbsoluteUri, but even after finding and fixing a couple of those, the problem didn’t go away. 

What was even more odd was that sometimes the same image would work, and sometimes it wouldn’t. What I needed was more information about exactly what was in the 403 error. I started up my trusty copy of Fiddler (and shut off TweetDeck and any other apps sending out network traffic), and started the upload process.

The upload process sends images from a WPF client app to blob storage. But it gets the SharedAccessSignature from a web service which resides on Windows Azure. The timeout on the SharedAccessSignature was set to 60 minutes, which should have been more than long enough for each one of these very small images. This seems like a horribly complicated process, but for binary files, POSTing directly to Azure Blob storage instead of going through a intermediary WCF service gave us an observed 40% speed increase. (your performance may vary).

Fiddler told me what the problem was right away:

The SharedAccessSignature maximum timeout cannot be more than 60 minutes.

What? I set it to exactly 60 minutes. How could it be more than 60? Well, I can think of two ways this could happen:

1) Here’s the code where I was setting up the SharedAccessSignature:

   1: var readPolicy = new SharedAccessPolicy
   2:  {
   3:      Permissions = SharedAccessPermissions.Read | SharedAccessPermissions.Write,
   4:      SharedAccessExpiryTime = DateTime.UtcNow + TimeSpan.FromMinutes(SharedAccessSignatureMaxTimeout)
   5:  };

where the SharedAccessSignatureMaxTimeout = 60

DateTime.UtcNow isn’t the most accurate call in the world. Potentially, this could lead to a SharedAccessExpiryTime which was just slightly greater than the max. The ticks property of the DateTime is probably more accurate.

2) If the System Times between the Storage Host, the Service Host and the Client are slightly off (something that I would expect to be quite common), I could see the scenario where it would seem like the expiry time appeared to be greater than the one hour allotted. With the large number of files we were sending up, I suspect we just hit this issue more than someone who was uploading a single file.

The solution, in either case, is to reduce SharedAccessSignatureMaxTimeout to be well short of 60 minutes. As soon as we did this, everything worked fine.

By Joe Beernink on May 27, 2011

I’m sure you are all creating unit tests to verify the output of the code you are writing to allow constant regression testing before any check in. But one of the other really nice things about having unit tests is the ability to performance test them when things are running more slowly than you expect.

Here’s how to do it:

  1. Run your list of unit tests in VS2010
  2. View the Test Results Window
  3. Sort the unit tests by duration. If one (or more) is really slow, Right Click on the test.
  4. Click “Create Performance Session”
  5. Select your sampling method and rerun the test.
  6. You’ll need to become familiar with the performance test outputs, but the key thing to look for is the “Hot Path”. Those lines are being called a lot, or taking up the majority of the time.
  7. Fix your code, and rerun the tests to confirm that the results are still correct, and that the fix is actually faster.
By Joe Beernink on April 26, 2011

We recently upgraded all of our Windows Azure applications from The Windows Azure SDK 1.2 to the Windows Azure SDK 1.4.  There are a lot of significant changes between the 1.2 and 1.4 release, including better diagnostics support, startup tasks and my personal favorite, the ability to Remote Desktop onto your roles.

In order to Remote Desktop onto your role, you need to configure the role to accept Remote Desktop connections.  The easiest way to do this is to right click on your Azure Cloud Project in Visual Studio 2010 and click Publish.  This will bring up the following dialog box:

AzureRemoteDesktop1

Now click on the Configure Remote Desktop Connections, and you should see the screen below:

AzureRemoteDesktop2

Click the ‘Enable connections for all roles’ check box, and then fill in a user name and password.  Click OK.  You should then be directed back to the first dialog box.  Since we deploy to Azure from our build servers and developers are never allowed deploy directly from their desktops, I select the ‘Create Service Package Only’ and click Ok.  This will update your role configurations with the appropriate settings to enable RDP. If you now save your changes, check in to source control, run the build and deploy that package to Azure, you should be able to RDP on to the role once it starts.

A couple of things to note:

1. If the role does not get to the ready status, you will not be able to RDP to it.  Usually, if we are having a deployment issue, it’s either that we have a bug in one of our startup tasks (usually a DOS cmd script), or a configuration issue with the app config.

2. Adding RDP will add another endpoint to your deployment, and you will not be able to upgrade or swap deployments with an existing role.  This is a huge inconvenience for us due to having SQL Azure Firewall rules and DNS entries set up for our roles. It means that adding RDP to a production system may result in down time for our users if we are not extremely careful about how we implement the change.  For that reason (and a few others), we have separate Azure Projects for our QA and our Production sites, and only enable RDP for our QA sites.

3. If you are having problems getting your role to start and you have complicated deployment startup scripts, the easiest thing to do is to comment out all the lines of your script, enable RDP and then deploy.  If the role starts, RDP onto the role, and open up your command script and try running each step on the virtual machine.  This approach isn’t 100% fool-proof, but having the ability to do this is a godsend compared to the old days of having to call up Azure Support to have them investigate the issue for you.  You also can check the event log and any other local logs, which can save you a lot of time as well.

There are lots of other significant issues you may experience when upgrading from 1.2 to 1.4.  It took us the better part of a week to upgrade our applications, and there are still differences between upgraded apps and new ones in the way that the templates for the applications are set up. In retrospect, it probably would have been easier to create new Azure projects and migrate our old code into them instead of trying to upgrade, but our code base dates back to the CTP 1 release of the Azure SDK, and perhaps those apps created since then will not be vulnerable to the same set of issues. Regardless, upgrading to 1.4 is something well worth the effort when all is said and done.

By Joe Beernink on March 23, 2011

Last week we were working on migrating one of our long running and successful applications that runs on Windows Azure from Visual Studio 2010 (ASP.NET MVC 2) and .NET 3.5 to Visual Studio 2010 (ASP.NET MVC 2) and .NET 4.0.  The automated upgrade process in Visual Studio is pretty easy to do, but at the end of the process, we could not get the app to run on Windows Azure.  It ran fine on our local Dev Fabric, and even outside of the Dev Fabric, but would get into the dreaded ‘Initializing / Busy’ Loop when starting on Azure.  This is usually a sign that the app is throwing an exception, either in the WebRole.cs or in the ApplicationStart in the Global.asax.  We were still running the Windows Azure SDK Release 1.2, since we were trying to change as few things as possible and this app was running just fine under 1.2, so we were pretty sure the problem had to do with a dll it couldn’t find, or a configuration issue with the web site.

Unfortunately, when you run into something like this, you can’t get much detail from either the system diagnostics or from your own logging because the problem happens so early in the application start-up cycle.  We called in the big guns on the Windows Azure support team.

It turns out that on our dev machines, in the machine.config, .NET defines a sectionGroup for <system.web.extensions>.  But on Azure, this definition does not appear in the machine configuration.  You have to add this to your own web.config files explicitly if you want to use the extensions.

Here’s the error that showed up when the Microsoft tech logged in to the VM and hit our web URL.

The configuration section 'system.web.extensions' cannot be read because it is missing a section declaration

126:     </system.web>

  127:   <system.web.extensions/>

  128:   <!--

It turns out, we weren’t actually using the the system.web.extensions namespace for anything, so removing it from our config file fixed the problem immediately.

We were probably more vulnerable to this problem because we section off our web config files into separate pieces so we can have test, QA and production versions of our config files, and those don’t automatically get updated during the upgrade process.  There were a few other instances where the DLL versions didn’t get updated, but this was the hardest to find because the setting was squirreled away in the machine.config.

Hope this saves someone some time.

By Joe Beernink on December 09, 2010

We’ve had an ongoing, no-repro bug in one of our apps that would pop up once a month or so since the dawn of time.  Users would attempt to register on the web site, get partway through, including entering their credit card numbers, but their status would never get changed from Pending to Registered.  The app would crash with a TimeOut Error while updating the status flag on the database.  This is beyond annoying, since the SQL statement is just updating a single field.  That should never time out.  We had never been able to figure this out, and have added dozens of lines of logging to try to get to the bottom of it.

At Microsoft’s Professional Developer’s Conference (PDC) last month, I sat in on this session, which showed the Microsoft Tier Interaction Profiler in action.  When one of our developers came to me to say this problem was once again rearing its ugly head, I remembered this presentation. 

We fired up the Web site in VS2010 using the Performance Analyzer with Tier Interaction Profiling turned on.  We then ran through the registration process on the site, and then stopped the profiler, and started looking through the results.  It took a moment to get to the right spot, but eventually we found the drop down that allowed us to switch to the TIP View.  Under that, we found the page we wanted to see, and at the bottom of the screen it showed us every SQL statement the code was executing.

screenshot (2)

We then found the Update Statement that was running against the database

UPDATE [Attendee].[Attendee]

SET [StatusID] = @p30, [UpdatedDate] = @p31

WHERE ([AttendeeID] = @p0) AND ([EventID] = @p1) AND ([StatusID] = @p2) AND ([AttendeeTypeID] = @p3) AND ([GroupID] = @p4) AND ([FirstName] = @p5) AND ([LastName] = @p6) AND ([Title] = @p7) AND ([CompanyName] = @p8) AND ([StreetAddress1] = @p9) AND ([StreetAddress2] = @p10) AND ([City] = @p11) AND ([StateProvinceID] = @p12) AND ([PostalCode] = @p13) AND ([Region] = @p14) AND ([CountryID] = @p15) AND ([EmailAddress] = @p16) AND ([WorkPhone1] = @p17) AND ([WorkPhone2] = @p18) AND ([Fax] = @p19) AND ([Website] = @p20) AND ([BusinessTypeID1] = @p21) AND ([BusinessTypeID2] IS NULL) AND ([LanguageID] = @p22) AND ([DateStamp] = @p23) AND (NOT ([Deleted] = 1)) )

Whoa.  Dude.  What the heck is that?  We’re updating 2 fields, yet every field in the Attendee table is listed in the where clause.  Not cool.  There is only one index on the table, and that’s on the AttendeeId field.  The presence of all these other fields removes any indexes from the update.  This app is written with Linq To SQL, and Linq To SQL is trying to use these fields as a concurrency check to make sure we are not overwriting someone else’s changes that may have been made since the record was selected from the database.  But we also know that we have our own way of concurrency tracking via the UpdatedDate field. 

We went into the DBML for the Attendee model, and changed the Update Check flag on all the fields of the Attendee table to Never, except for the Attendee Id, and the UpdatedDate, and reran the profile.  After that we ended up with the following statement instead.

UPDATE [Attendee].[Attendee]

SET [StatusID] = @p2, [UpdatedDate] = @p3

WHERE ([AttendeeID] = @p0) AND ([UpdatedDate] = @p1)

The query was roughly twice as fast as the original.

Then we added an index to AttendeeId + UpdatedDate to the table, and reran the profile.  The query was now 10 times as fast as the original on our test database, and infintely faster on the very large production database.  The problem is now resolved.

I highly encourage everyone to get familiar with this tool. It’s free for VS2010 Ultimate Users.  If you’re not using it, you’re missing out a great way to save time solving problems.  This whole investigation using this tool took us about 15 minutes.  Knowing that we have previously spent hours trying to track this intermittent issue down on this I say ZOMG WOW!

By Joe Beernink on November 03, 2010

This year’s Microsoft PDC was greatly reduced in both the physical size, and in the scope of the announcements made compared to the last two years.  In 2008 at PDC, Microsoft Announced Windows Azure.  In 2009, we got our first look at  Silverlight 4, RIA Services and ASP.NET MVC 2.  This year, the focus was on Windows Azure and Windows Phone 7.  There were some in depth looks at ASP.NET MVC 3, and some new tooling support in Visual Studio for Azure and Testing, but there weren’t a lot of ‘stunning’ announcements.

For me, attending PDC is so much more than just watching the sessions.  Though the days are exhausting, the sessions are energizing, and fill my head with new ideas.  You get to meet many of the Microsoft Developers and many of the Open Source Developers who are working on the next versions of the tools we developers use every day.  They listen to the feedback we provide, and answer the relentless flow of questions from attendees.  And if they don’t know the answer, they follow up with you the next day via email with the answer.

The last two PDC’s have had a profound impact both on the future of Palador, and on my own personal career.  In 2008, even though we didn’t attend, the launch of Azure and our early commitment to get involved in the platform has pushed us to become well recognized by the Azure Team.  My attendance last year resulted in great contacts within the industry, many of whom were there again this year. I talked for a while on Friday with one of them about the old days – our days as members of the Windows Azure TAP program – when everything was new and everything we did took a lot of hard work.  What the Azure Team has now provided to users almost seems like cheating.  New users may take tools such as an improved Portal and SCOM for the Windows Azure Platform for granted.  But those of us in the TAP will look at them and laugh to ourselves and wonder how we did what we did when we did it.

By the third session this year, I had started a new page in my notebook, and started writing out new technical goals for both Palador and for myself.  Most of the goals revolve around educating everyone on the new tools available, like Intellitrace, The Razor View Engine for ASP.NET MVC 3, SQL Azure Data Sync, SQL Azure Reporting, Azure App Fabric Caching and a half dozen others.  Some of the goals involve looping back through the applications we have already launched and updating them to take advantage of new features of Azure or ASP.NET MVC.  And some involve trying to figure out where we will position ourselves in new areas like Windows Phone 7 development.

One thing is for sure.  PDC is much more than just a series of technical sessions.  It’s a great step in the journey of keeping current in what is going on in the software development world, and it really helps us to set the direction for the future.  And that’s how we’re staying ahead of the curve, and the competition.

By Joe Beernink on September 28, 2010

Recently, we built an ASP.NET MVC 2 web site for a customer.  Then they asked us to build another one, almost the same, but with enough differences in both the data model, the design and flow that we branched the code and created a new version of it.  The original site was built in VS2008, and the new one was upgraded to VS2010.

A few weeks later, they came back to us again, and asked for three more sites, just like the second one, but with a few differences.  The data model held pretty true, but the page flow was different, and they wanted a portal like presentation, where a user could easily switch between any of the four sites.  The sites were to be set up as follows:

www.companyname.com  (Root of Portal)

www.companyname.com/ProductA

www.companyname.com/ProductB

www.companyname.com/ProductC

www.companyname.com/ProductD

We obviously didn’t want to make four copies of the code since the sites had to feel like one site at the portal level.  And we couldn’t very well go putting case statements in every page to trap all the differences.  Most of the pages and processes for each product are the same, but a few are different.  We needed to come up with an approach that allowed us to share as much as we could, and then only create new functionality where we absolutely had to.

We came up with the idea of the abstract controller.  We created an Abstract ProductController that defined abstract methods for all of the Actions needed for the Products.    We then added protected methods inside the AbstractController that implemented the Actions, and returned views.

The next step was to create a Product specific controller for each of the Products that implemented all of the methods of the base controller.  Most often, the Product specific controller simply called the Abstract Controller’s protected methods, but where the logic differed, it was free to specify its own logic.  This cut down development and testing effort tremendously.

Here is shortened code for the abstract controller and a sample Product Controller

   1: public abstract class ProductController : BaseController
   2: {
   3:     #region virtual Methods that need to be implemented for each Product Controller
   4:     public abstract ActionResult Home();
   9:     public abstract ActionResult Faqs();
  13:     #endregion
  14:  
  15:     #region Base Methods
  16:     
  17:     protected ActionResult GetProductHomeView(string productQualifier)
  18:     {
  19:         var productId = ConfigurationManager.AppSettings.Get(string.Format("ProductId_{0}", productQualifier));
  20:         int localProductId;
  21:  
  22:         if (!Int32.TryParse(productId, out localProductId))
  23:         {
  24:             var homeViewModel = new HomeViewModel
  25:             {
  26:                 Title = "Welcome To Company WebSite",
  27:             };
  28:  
  29:             //show the big header
  30:             return View(StringConstants.ActMainHome, homeViewModel);
  31:         }
  32:  
  33:         var productViewModel = new ProductViewModel
  34:         {
  35:             Title = "Welcome To Product",
  36:             ProductId = localProductId,
  37:         };
  38:  
  39:         ViewData[StringConstants.VdProductId] = localProductId;
  40:         ViewData[StringConstants.VdProductQualifier] = productQualifier;
  41:         return View(StringConstants.ActProductHome, productViewModel);
  42:     }
  43:  
  44:     protected ActionResult GetFaqsView(string productQualifier)
  45:     {
  46:         var productId = ConfigurationManager.AppSettings.Get(string.Format("ProductId_{0}", productQualifier));
  47:         int localProductId;
  48:  
  49:         if (!Int32.TryParse(productId, out localProductId))
  50:         {
  51:             var homeViewModel = new HomeViewModel
  52:             {
  53:                 Title = "Welcome To Company Web Site",
  54:             };
  55:  
  56:             //show the big header
  57:             return View(StringConstants.ActMainHome, homeViewModel);
  58:         }
  59:  
  60:         var faqViewModel = new FaqViewModel
  61:         {
  62:             Title = "FAQ",
  63:             ProductId = localProductId,
  64:         };
  65:         ViewData[StringConstants.VdProductQualifier] = treatmentProductQualifier;
  66:         ViewData[StringConstants.VdProductProgramId] = localProductProgramId;
  67:         return View(StringConstants.ActTreatmentFaqs, faqViewModel);
  68:     }
  69: }
  70:  
  71: public class ProductAController : ProductController
  72: {
  73:     public const string ProductQualifier = "ProductA";
  74:  
  75:     [HttpGet]
  76:     public override ActionResult Home()
  77:     {
  78:         return GetProductHomeView(ProductQualifier);
  79:     }
  80:  
  81:     [HttpGet]
  82:     public override ActionResult Faqs()
  83:     {
  84:         return GetFaqsView(ProductQualifier);
  85:     }
  86: }

A few notes:

1.  The ProductQualifier is a critical concept.  When the Name Product Controller is instantiated, this value is passed into the protected calls to help identify the correct resources to load (i.e. this is how the product is identified).  But we don’t explicitly trust this string.  We make sure (by checking the config file), that we are actually set up to handle this product.  If we somehow get past the controller and pass in an invalid product, the user is redirected to the Portal Home Page.  This ensures that no process is ever run without picking a valid product.

2.  The Views must be placed in the Shared Views Folder, otherwise the MVC View engine cannot find them.

3.  Each view that references the controllers, either for a form submit, or a redirect, must know explicitly which controller to submit to.  Fortunately, this is already available in the MVC framework.  For form submits, you should always submit to the same controller that served up the page.  The following line of code works quite nicely.

   1: <% using (Html.BeginForm(StringConstants.ActBuyProduct, Url.RequestContext.RouteData.GetRequiredString("Controller"), FormMethod.Post, new { name = StringConstants.ActBuyProduct }))

 

4. On The Portal View, where we want to explicitly switch to different controllers, it’s pretty straightforward.  Note that the Action Method is exactly the same.  The controller changes.

   1: <%= Html.ActionLink("Product A", StringConstants.ActProductHome, StringConstants.CtrlProductA)%><br />
   2: <%= Html.ActionLink("Product B", StringConstants.ActProductHome, StringConstants.CtrlProductB)%><br />
   3: <%= Html.ActionLink("Product C", StringConstants.ActProductHome, StringConstants.CtrlProductC)%><br />
   4: <%= Html.ActionLink("Product D", StringConstants.ActProductHome, StringConstants.CtrlProductD)%><br />

 

5.  We use named constants for our string names everywhere.  We’re pretty religious about it, and that made this conversion and refactoring a lot easier, especially when done with ReSharper.  I didn’t define any of the constants in the sample above, but they should be pretty easy to figure out.

This solution works pretty well where the number of Products is well known, and the functionality for each is similar, but not exact.  You will have to add code and configuration each time you add a new Product.  However, with the requirements of this project, we had to add code each time anyway to handle product process changes, and the savings of this simple, common approach more than compensated for the fact that the solution may not be perfect from a purist’s view.  We’re not talking a few hours of saved time here.  We’re talking hundreds of hours of dev time saved.  All from a simple little pattern.

By Joe Beernink on June 15, 2010

The systems development world today is all about instant gratification.  Web search results must return in 1.2 seconds or less.  Web pages must load in less than 3 seconds.  Search results not only must find every result for ‘Flugenschnitzel’, it must show the most popular references to ‘Flugenschnitzel’ in the last 30, 60 or 90 days.  All of these things matter to users.  People want the most up to date information possible, and they want it now.

But what do you do what wanting up to date and means that the page take too long to load?  You make an intelligent choice. 

For instance, if you are calculating the most popular product on a site, if the ranking of that product is five minutes old, it usually doesn’t matter.  If you are calculating the remaining amount in a bank account, it does matter, so the following approach doesn’t apply, and the user may need to wait a fraction of a second longer because they need to see absolutely accurate information all the time.

We recently had scenario A.  There were three lists of products on a page, sorted by Most Recently Updated Products, Most Popular Products, and Featured Products.  The page as originally written and tested here on our development servers took no time at all to load.  But when we moved the site to Windows Azure, and put it under load, performance dropped dramatically.   It didn’t take long to understand why.  There was a greater latency between the client and the server and the server and the database.  Each round trip we were making to the database or to the server came at a cost.  We couldn’t control the round trip latency, but we could control the number of trips. We started to tune the result set to reduce the amount of data needed for each list.  We improved our client side caching so that fewer graphics were being brought down. 

But one query kept slowing down the page refresh, and it got worse as more data was added to the site.  It was the Most Popular Product query.  What was happening was that every time the page refreshed, the popularity of all records was being calculated.  Obviously not a good idea now, but when we laid out the site, it was a minor query, and a minor part of the site.  We didn’t tune it until we needed to, which is arguably a good thing.

We considered our options.  We could have the popularity update every time someone used or viewed the product, but that would cause those actions to be slower.  We could cache the results for a few minutes on the server side, but someone would have to pay the price for the slow query, and that would also be bad.

We decided to create a new worker role in Azure that wakes up every few minutes, calculates the popularity (and a few other key metrics), and then goes back to sleep.   The worker role runs out of sight of the user, and populates a flattened metrics table that is easily and quickly queried.  The views used by the MVC application never had to change, but we did change the data access layer to use these new fields.  It took about four hours to build and deploy the new Accumulator, and as soon as the deployment was complete and the stats updated, the site went from 7 seconds for the home page refresh to less than 1 second.  The users never know, and rarely care that the list is a few minutes old.  They want the instant gratification of a quick page load.

Doing batch programming like this isn’t anything new.  And although the solution was quick and easy using an Azure worker role, it could be a little better if it were a scheduled job instead of one that was always running and sleeping.  The Azure platform charges by CPU usage, so it would be really nice if the job wasn’t always there running up the charges.  But in this circumstance, we had a site that was unusable due to performance, and the ability to deploy this type of solution quickly really kept the project on time and let us focus on enhancing the user experience instead of worrying about infrastructure.

By Joe Beernink on May 07, 2010

We’ve been working with Shared Access Policies in Azure for the last week or so, and for the most part it was working.  But it would only work for a while, then it would stop.  Then it would start working again.   It took some help from Steve Marx and Jai Haridas on the Azure Team to figure out what was going on.

This first piece of code is part of our storage manager that retrieves the Uri for the blob from Azure Blob Storage, and creates a Shared Access Policy for that blob so that our Silverlight Video Player can directly access the blob without going through a very slow web service call.

   1: /// <summary>


    
   2:         /// Get the Uri for a specified video blob


    
   3:         /// </summary>


    
   4:         /// <param name="videoId">The unique identifier of the video</param>


    
   5:         /// <returns>A Uri ponting to the Video</returns>


    
   6:         public Uri GetVideoBlobUri(Guid videoId)


    
   7:         {


    
   8:             var log = Log4NetHelper.GetLogger();


    
   9:             log.DebugFormat("Getting Video for video id {0}", videoId);


    
  10:  


    
  11:             try


    
  12:             {


    
  13:                 CloudBlobContainer container = GetContainer("VideoContainerName");


    
  14:  


    
  15:                 CloudBlockBlob cloudBlockBlob = container.GetBlockBlobReference(string.Format(CultureInfo.InvariantCulture, "{0}", videoId));


    
  16:                 


    
  17:                 var readPolicy = new SharedAccessPolicy


    
  18:                 {


    
  19:                     Permissions = SharedAccessPermissions.Read,


    
  20:                     SharedAccessExpiryTime = DateTime.UtcNow + TimeSpan.FromMinutes(10)


    
  21:                 };


    
  22:  


    
  23:                 var blobUri = new Uri(cloudBlockBlob.Uri.AbsoluteUri + cloudBlockBlob.GetSharedAccessSignature(readPolicy));


    
  24:  


    
  25:                 log.DebugFormat("GetVideoBlobUri successfully retrieved for video id {0}", videoId);


    
  26:  


    
  27:                 return blobUri;


    
  28:             }


    
  29:             catch (StorageClientException ex)


    
  30:             {


    
  31:                 // If the blob was not found, return null
  32:                 if (ex.ErrorCode == StorageErrorCode.BlobNotFound)
  33:                     return null;
  34:                 // Rethrow the exception in all other cases
  35:                 throw;
  36:             }
  37:             catch (Exception ex)
  38:             {
  39:                 log.ErrorFormat("Error while retrieving video blob {0}", ex.Message);
  40:                 throw;
  41:             }            
  42:         }

In our Silverlight Web Service, we call this manager class and return the Uri embedded in an XElement.  Originally we did this to allow us to return other information with the Uri, but at this point, the Uri is all we are passing through.

   1: /// <summary>
   2: /// Get video URI.
   3: /// </summary>
   4: /// <param name="videoId">ID of the given video.</param>
   5: /// <returns>Status</returns>
   6: public XElement DemoVideoUri(string videoId)
   7: {
   8:     var blobManager = UnityFactory.Current.Resolve<IBlobManager>();
   9:     var blobUri = blobManager.GetVideoBlobUri(new Guid(videoId));
  10:  
  11:     string xml = string.Format("<VideoUri>{0}</VideoUri>", HttpUtility.HtmlEncode(blobUri.AbsoluteUri));
  12:     var sr = new StringReader(xml);
  13:     return XElement.Load(sr);
  14: }

On the Silverlight side, we take make a call to this web service, extract the Url from the XElement, and pass the Url into the MediaPlayer like so

   1: var client = new WebClient();
   2:  
   3: var videoUri = GetVideoUri();
   4:  


    
   5: client.DownloadStringCompleted += (x, y) =>


    
   6:                                Dispatcher.BeginInvoke(


    
   7:                                    () =>


    
   8:                                    {


    
   9:                                        var xdoc = XDocument.Parse(y.Result);


    
  10:                                        var query = from b in xdoc.Descendants()


    
  11:                                                    select b.Value;


    
  12:                                        HostedVideoUri = new Uri(HttpUtility.HtmlDecode(query.First()), UriKind.Absolute);


    
  13:                                        mediaPlayer.Source = HostedVideoUri;


    
  14:                                        Log(string.Format("HostedVideoUrl = {0}", HostedVideoUri.AbsoluteUri));


    
  15:                                    }


    
  16:                                    );


    
  17:  


    
  18: client.DownloadStringAsync(new Uri(videoUri, UriKind.Absolute));


    
  19:  


    
  20:  


    
  21: private static string GetDemoVideoUri()


    
  22: {


    
  23:     var host = HtmlPage.Window.Eval("window.location.hostname;") as string;


    
  24:     var path = HtmlPage.Window.Eval("window.location.pathname;") as string;


    
  25:     var refererUri = host + path;


    
  26:     var url = string.Format("http://{0}/xxxx.Svc/DemoVideoUri/{1}", GetHost(), VideoId);


    
  27:     return url;


    
  28: }

This is all pretty simple code (and the above code works, by the way).

Where we ran into problems was in the Web Service, when populating the xml string.  Originally, we used blobUri.ToString() instead of blobUri.AbsoluteUri.   This caused big issues (403 errors returned from the Azure Storage Service), when the Video Player tried to retrieve the blob because the Url returned from the Shared Access Policy generator can have spaces in it.  And Uri.ToString and Uri.AbsoluteUri work very differently when handling spaces.  I did not know this until last night.  Uri.ToString unescapes the Uri before returning it.

So why did it work sometimes, and not all the time?  Simple.  Sometimes the Uri from the Shared Access Policy Generator has spaces, and sometimes it does not.  The former did not work, while the latter did.  We were looking for a pattern in the number of times we called the web service, or the interval between calls, and the size of the video blob.  But it was as simple as a single space.

I think we spent at least 8 hours over the last two days trying to track down this bug.  Hopefully this saves someone else some grief.