Posted on 6. March 2014

‘Ghost’ Web Resources & Client Metadata Caching

When I blogged about my CRM 2013 Start Menu solution I said I would also post about how it caches metadata on the client – so here is that post!

Metadata is the information that describes other data and in the case of CRM 2013 metadata describes entities, relationships and attributes as well as other configuration elements such as the sitemap. The Start Menu solution needed to read the sitemap in order to dynamically display it in the drop down command bar button. Initially the solution read the site map from the server every time the menu was displayed. This wasn't too bad but the solution also then had to make additional requests for entity metadata to retrieve the localised display name, image and object type code. What's more is the solution then had to retrieve the user's privileges and iterate over the sitemap to decide if the user had access or not. This is a common scenario with any webresource development for Dynamics CRM – it could be option set labels or view layoutxml - all need by the client every time the page is displayed. Since this metadata doesn't change very often it makes it a very good candidate for caching.

Caching isn't the problem

Whenever designing a cache solution – the first thing to think about is how to invalidate that cache. It's no good being able to store something for quick access if you can't tell if it is stale and needs to refresh – this could lead to problems much worse than poor performance!

Dynamics CRM neatly already provides us with a client side caching mechanism it uses for Web Resources. I blogged about this back in the CRM 2011 days – but it really hasn't changed with CRM 2013. The general principle is that if you request a Web Resource in the following format then you will get the HTTP headers from the server that means that the browser/proxy server can cache the file.

http://server/%7B000000000000000000%7D/WebResources/SomeFile.js

The net result is that the next time that the browser requests this file, provided it has the same number before the WebResources folder then the file will not be requested from Dynamics CRM but served from the cache.

Every time you publish a web resource the number that is used to request web resources is changed so that the client browser gets a new copy and then caches that until the next change of number.

So we have a very effective caching mechanism for static content – but how do we make use of this for dynamic content? What we need is a way of storing the sitemap and all the other metadata needed into a web resource so that it will be cached – but we don't want to have to update a webresource with this information – what we need is something similar to ASP.Net where we can dynamically generate the web resource when requested and then cache the result on the client.

Dynamic 'Ghost' Web Resources

The magic happens by a Plugin that is registered on Retrieve Multiple of webresource. When the Plugin detects that the request is for the SiteMap web resource, all this metadata is retrieved server side, converted into JSON and then added dynamically to the output collection as a webresource record. The user's LCID is also read from the web resource name and used to pick to correct localised labels. The interesting thing here is that the web resource being requested by the Javascript doesn't actually exist – they are 'ghost' web resources. If the plugin wasn't there, then the platform would return the usual 404 File not found but as long as the Plugin intercepts the query and provides an output then the platform doesn't mind that we are requesting something that doesn't exist. This technique provides a host of opportunities for adding additional request information to the name of the web resource that can then be used to determine the contents of the web resource. This allows the web resource contents to vary depending on the request:

  • Varying content by language/LCID
  • Varying content by record id
  • Varying content by user
  • Varying content by other parameters such as date

 

Check out the code for this web resource plugin to see how it's done.

Invalidating the cache

Since we now can cache our server side generated JSON on the client – we need to know how to clear the cache when something changes. In the case of the 'Start Menu' solution that something changing is the sitemap.xml or entity names. The cache key number that is used by the client will change whenever a web resource is added or updated so to clear the client cache we simply need to update a dummy web resource. Part of the solution publishing that contains a sitemap or entity name change should always include a web resource update so that the client will reflect the updates.

Be careful with caching of sensitive data

Caching of metadata is the most common use of this technique, but it could also be used for caching commonly used reference data such as products or countries. This cache can be invalidated easily by making a simple request for the most recent modified on date – but make sure you don't cache any sensitive data since this would be accessible by anyone with access to the client machine.

Making things easier

A future update of SparkleXRM will contain a client side metadata caching framework that I'm working on that uses the techique I describe here, but in the mean time I hope this helps you get better performance from your client side code.

@ScottDurow