Posted on 15. April 2014

Fiddler2: The tool that gives you Superpowers – Part 3

This post is the third post in the series 'Fiddler – the tool that gives you superpowers!'

Faster than a Speeding Bullet

If you have done any development of Web Resources with Dynamics CRM then I'm certain that you'll have become impatient whilst waiting to first deploy your solution and then publish it before you can test any changes. Everytime you need to make a change you need to go round this loop which can slow down the development process considerably. Using the Auto Responders I described in my previous post (Invisibility) you can drastically speed up this development process by using Fiddler to ensure changes you make to a local file in Visual Studio are reflected inside Dynamics CRM without waiting for deploying and publishing. You make the changes inside Visual Studio, simply save and refresh your browser and voilà!

Here some rough calculations on the time it could save you on a small project:

Time to Deploy

15

seconds

Time to Publish

15

seconds

Debug iterations

20

 

Number of web resources

30

 

Development Savings

5

hours

Time to reproduce live data in test/development

1

hour

Number of issues to debug in live

10

 

Testing Savings

10

hours

     

Total Savings for small project

15

hours

 

What is perhaps more important about this technique that it saves the frustration caused by having to constantly wait for web resource deployment and ensures that you stay in the development zone rather than being distracted by the latest cute kitten pictures posted on facebook!

Do remember to deploy and publish your changes once you've finished your development. It seems obvious but it is easily forgotten and you're left wondering why your latest widget works on your machine but not for others!

More information can be found on this at the following locations:

@ScottDurow

Posted on 15. April 2014

Fiddler2: The tool that gives you Superpowers – Part 4

This post is the fourth and final post in the series 'Fiddler – the tool that gives you superpowers!'

Ice Man

Perhaps Ice Man is the most tenuous super power claim but it's regarding a very important topic – HTTP Caching. Having a good caching strategy is key to having good client performance and not over-loading your network with unnecessary traffic. Luckily Dynamics CRM gives us an excellent caching mechanism – but there are situations where it can be accidently unknowingly bypassed:

  1. Not using relative links in HTML webresources
  2. Loading scripts/images dynamically without using the cache key directory prefix
  3. Not using the $webresource: prefix in ribbon/sitemap xml.

Luckily we can use Fiddler to keep our servers running ice cold by checking that files that are not being cached when they should be. There are types of caching that you need to look for:

Absolute expiration

These web resources will not show in Fiddler at all because the browser has located a cached version of the file with an absolute cache expiration date and so it doesn't need to request anything from the server. By default CRM provides an expiration date of 1 year from the date requested, but if the web resource is updated on the server then the Url changes and so a new version is requested. This is why you see a request similar to /OrgName/%7B635315305140000046%7D/WebResources/somefile.js. Upon inspection of the response you will see an HTTP header similar to:

HTTP/1.1 200 OK
Cache-Control: public
Content-Type: text/jscript
Expires: Tue, 14 Apr 2015 21:18:35 GMT

Once the web resource is downloaded on the client it is requested again until April 14 2015 unless a new version is published where CRM will request the file using a new cache key (the number between the Organization name and the WebResources directory). You can read more about this mechanism on my post about web resource caching.

ETAG Cached files

These resource are usually images and static JavaScript files that are assigned an ETAG value by the server. When the resource changes on the server it is assigned a different ETAG value. When the browser requests the file it sends the previous ETAG value if it hasn't been modified then the server responds with a 304 response meaning that the browser can use the local cached file.

Files that use ETAG caching will show in grey in Fiddler with a response code of 304:

During your web resource testing it is a good idea to crack open Fiddler and perform your unit tests – you should look for any non-304 requests for files that don't need to be downloaded every time they are needed.

Another way to ensure that your servers are running cool as ice is to look at the request execution length. Occasionally code can be written that accidently returns much too much data than required - perhaps all attributes are included or a where criteria is missing. These issues don't always present themselves when working on a development system that responds to queries very quickly, but as soon as you deploy to a production system with many users and large datasets, you start to see slow performance.

There are a number of ways you can test for this using Fiddler:

Visualise Request Times

The order in which your scripts, SOAP and REST requests are executed in can greatly affect the performance experienced by the user and so you can use Fiddler's Time line visualizer to see which requests are running is series and which are running in parallel. It also shows you the length of time the requests are taking to download so that you can identify the longest running requests and focus your optimisation efforts on those first.

  •  

    Simulate Slow Networks

    If you know that your users will be using a slow network to access CRM or you would just like to see how the application responds when the requests start to take longer because of larger datasets you can tell fiddler to add an artificial delay into the responses. To do this you can use the built in Rules->Performance->Simulate Modem Speeds but this usually results in an unrealistically slow response time. If you are using Auto Responders you can right-click on the Rule and use set Latency – but this won't work for Organization Service/REST calls. The best way I've found is to use the Fiddler Script:

    1) Select the 'Fiddler Script' Tab

    2) Select 'OnBeforeRequest' in the 'Go to' drop down

    3) Add the following line to the OnBeforeRequest event handler.

    This will add a 50 millisecond delay for every kB requested from the server which assuming there was no time server time would result in ~160 kbps downloads.

    If you've not used Fiddler for your Dynamics CRM Development yet I hope these posts are enough to convince you that you should give it a try – I promise you'll never look back!

    @ScottDurow

    Posted on 15. April 2014

    Fiddler2: The tool that gives you Superpowers - Part 2

    This post is the second post in the series 'Fiddler – the tool that gives you superpowers!'

    Invisibility

    This time it's the superpower of Invisibility! Wow I hear you say!

    Fiddler is a web debugger that sits between you and the server and so is in the unique position of being able to listen for requests for a specific file and rather than returning the version on the server return a version from your local disk instead. This is called and 'AutoResponder' and sounds like a super-hero it's self – or perhaps a transformer (robots in disguise).

    If you are supporting a production system then the chances are that at some point your users have found an issue that you can't reproduce in Development/Test environments. Auto Responders can help by allowing us to update any web resource (html/JavaScript/Silverlight) locally and then test it against the production server without actually deploying it. The Auto Responder sees the request from the browser for the specific web resource and rather returning the currently deployed version, it gives the browser your local updated version so you can test it works before other users are affected.

    Here are the steps to add an auto responder:

    1) Install Fiddler (if you've not already!)

    2) Switch to the 'Auto Responders' tab and check the two checkboxes 'Enable automatic responses' and 'Unmatched requests pass-through'

    3) To ensure that the browser requests a version of the web resource rather than a cached version from the server you'll need to clear the browser cache using the convenient 'Clear Cache' button on the tool bar.

    4) You can ensure that no versions get subsequently cached by selecting Rules-> Performance-> Disable Caching.

    5) You can now use 'Add Rule' to add an auto responder rule. Enter a regular expression to match the web resource name

    regex:(?insx).+/<Web Resource Name>([?a-z0-9-=&]+\.)*

    then enter the file location of the corresponding webresource in your Visual Studio Developer Toolkit project.

    You are now good to go so that when you refresh your browser the version of your web resource will be loaded into the browser directly from your Visual Studio project. No need to publish a file to the server and affect other users.

    There is one caveat to this – If the script that you are debugging updates data then this approach is probably not a good idea until you are have fully tested the script in a non-production environment. Only once you have QAed and ready to deploy can be it be used against the production environment to check that the specific user's issue is fixed before you commit to deploying it to all users.

    Read the next post on how to be faster than a speeding bullet!

    @ScottDurow

     

    Posted on 15. April 2014

    Fiddler2: The tool that gives you Superpowers – Part 1

    The next few posts are for those who saw me speaking at the most recent CRMUG UK Chapter meeting about Fiddler2 and wanted to know more (and as a thank you to those who voted for me in X(rm) factor!). I've been using Fiddler for web debugging for as long as I can remember and I can honestly say that I could not live without it when Developing Dynamics CRM extensions as well as supporting and diagnosing issues with existing solutions. I first blogged about it in connection with SparkleXRM development but this post elaborates further on the super powers it gives you!

    What is a Web Debugger?

    Fiddler2 is a Web Debugger which basically means that it sits between your browser and the server just like any normal proxy, but the difference is that it shows you all the HTTP traffic going back and forwards, allows you to visualise it in an easy to read format as well as allowing you to 'Fiddler' with it – hence the name.

    You can easily install fiddler for free by downloading it from http://www.telerik.com/fiddler.

    The following posts describe the superpowers that Fiddler can give you whilst you are developing solutions or supporting your end users.

    X-Ray Vision

    When you perform any actions in your browser whilst Fiddler is running then each and every request/response is being logged for your viewing pleasure. This log is incredibly useful when you need to see what requests your JavaScript or Silverlight is sending to the server. It shows you the error details even when the user interface may simply report that an 'Error has occurred' without any details. The prize for the most unhelpful error goes to Silverlight with its 'Not Found' message – the actual error can only be discovered with a tool like Fiddler2 by examining the response from the server to see the true exception that is hidden by Silverlight. The HTTP error code is your starting point and Fiddler makes it easy to see these at a glance by its colour coding of request status codes - the most important of which are HTTP 500 requests that are coloured red. For any solution you are developing, the bare minimum you should look at is for any 404 or 500 responses.

    If you wanted to diagnose a problem that a user was having with CRM that could not reproduce then try following these steps:

    1. Ask the user experiencing the issue to install Fiddler2 (this may require administrator privileges if their workstation is locked down).
    2. Get to the point where they can reproduce the problem – just before they click the button or run the query, or whatever!
    3. Start Fiddler
    4. Ask the user to reproduce the issue
    5. Ask the user to click File->Save->All Sessions and send you the file.
    6. Once you've got the file you can load it into your own copy of Fiddler to diagnose the issue.

    If the user has IE9 or above and they are not using the outlook client then the really neat thing about the latest version of Fiddler is that it can import the F12 Network trace. This allows you to capture a trace without installing anything on the client and then inspect it using Fiddlers user interface. To capture the network traffic using IE:

    1. Get to the point where they are about to reproduce the issue
    2. Press F12
    3. Press Ctrl-4
    4. Press F5 (to start the trace)
    5. Reproduce the issue
    6. Switch back to the F12 debugger window by selecting it
    7. Press Shift-F5 to stop the trace
    8. Click the 'Export Captured Traffic' button and send you the file

    Now you can load this file into fiddler using File->Import Sessions->IE's F12 NetXML format file.

    Once you found the requests that you are interested in you can then use the inspectors to review the contents – the request is shown on the top and the response is shown on the bottom half of the right panel. Both the request and response inspectors gives you a number of tabs to visualise in different ways depending on the content type. If you are looking at JavaScript, HTML or XML your best bet is the SyntaxView tab that even has a 'Format Xml' and 'Format Script/JSON' option on the context menu. This is great to looking at SOAP requests and responses that are sent from JavaScript to make sure they are correctly formatted.

    The following screen shows a soap request from JavaScript and inspectors in syntax view with 'Format Xml' selected.

  •  

    This technique is going to save you lots of time when trying to work out what is going on over the phone to your users!

    Next up is Invisibility!

    @ScottDurow