Solving Three Problems in Windows Azure Diagnostics

Hi Guys,

I want to walk through three issues you can have with Windows Azure Diagnostics and a Worker Role. I assume you want to access the Windows Azure Trace Logs, since you use the Trace command to write out exceptions and status messages in the onStart code.

e.g, Trace.WriteLine(“Starting my service.”)

Also you have the WAD trace listener on, which it is by default.

https://gist.github.com/1757147

You might have a custom worker role that starts some sort of background service and perhaps it fails immediate due to some sort of configuration.

Symptoms

You notice the role keeps recycling and recovering.

On Demand Transfer or Schedule Transfer of Diagnostics logs do not work at all, so you cannot get any Trace Information whatsoever.

Solution

Put this method in your WorkerEntryPoint.cs file and call it at the beggining of OnStart() and in any Catch Exception block

https://gist.github.com/1757083

e.g Start

public override bool OnStart()
{
WaitForWindowsAzureDiagnosticsInfrastructureToCatchUp();
try
{ 

e.g Exception

catch (Exception ex)
           {
               TraceException(ex);
               Trace.Flush();
               WaitForWindowsAzureDiagnosticsInfrastructureToCatchUp();
               return true;
           } 

You are able to diagnose the problem since On Demand Transfer/Scheduled Transfer works and you can then get to the trace logs to see error messages you have written to Trace. Recall that Windows Azure has a settings to automatically have a trace listener on to redirect trace to its WAD table.

Symptoms

You notice the role keeps recycling and recovering or event stars up but is unresponsive.

On Demand Transfer does not work – You try but it just does not complete or hangs

Below is screen shots of On Demand Transfers with the Cerebrata Diagnostics Manager.

Solution

If you cannot do an On Demand Transfer of trace logs, perhaps it keeps recycling and recovering to fast for a On Demand Transfer to occur. Then what you do is temporarily configure a Scheduled Transfer of the Trace Logs

If using Cerebrata Diagnostics Manager

Click Remote Diagnostics in the Subscriptions under your Data Sources

Once you have configured Schedules transfer, this tool will basically UPLOAD a configuration file into your BLOB container: wad-control-container

Azure will automatically detect changes in this container and apply them to the Diagnostics Manager. Hence configuration of Windows Azure Diagnostics On The Fly

Now that we have scheduled transfer in place REBOOT the role that is causing the issue and then wait for it to try start up and fail, and then just go download the trace logs and it should be there.

So, ensure you have a silly sleep command in your work entry point OnStart and in areas where you catch exceptions in case your worker role crashes before Windows Azure Diagnostics!

Try On Demand Transfers if there is an issue, and if that does not work,  configure a scheduled transfer on the fly and then reboot the role to get the start up logs.

Scheduled Transfers will impact your billing of Storage Services, MAKE SURE you turn it OFF when you finished diagnosing the issue, else you will get BILLED for it.

Notice in my screen shot I ALWAYS use a quota so I never over use diagnostics storage – and Windows Azure Trace Logs are stored in TABLE Storage:

Remember configuration of WAD is in Blob and the actual trace logs are in Tables.

Source: http://romikoderbynew.com/2012/02/07/windows-azurediagnosing-role-start-ups/