Posted on 11. March 2017

Simplified Connection Management & Thread Safety (Revisited)

There is one certainty in the world and that is that things don't stay the same! In the Dynamics 365 world, this is no exception, with new features and SDK features being released with a pleasing regularity. Writing 'revisited' posts has become somewhat of a regular thing these days.

In my previous post on this subject back in 2013 we looked at how you could use a connection dialog or connection strings to get a service reference from the Microsoft.Xrm.Client library and how it can be used in a thread safe way.

Microsoft.Xrm.Tooling

For a while now there has been a replacement for the Microsoft.Xrm.Client library – the Microsoft.Xrm.Tooling library. It can be installed from NuGet using:

Install-Package Microsoft.CrmSdk.XrmTooling.CoreAssembly

When you use the CrmServerLoginControl, the user interface should look very familiar because it's the same that is used in all the SDK tools such that Plugin Registration Tool.

The sample in the SDK shows how to use this WPF control.

The WPF control works slightly differently to the Xrm.Client ShowDialog() method – since it gives you much more flexibility over how the dialog should behave and allows embedding inside your WPF application rather than always having a popup dialog.

Connection Strings

Like the dialog, the Xrm.Tooling also has a new version of the connection string management – the new CrmServiceClient accepts a connection string in the constructor. You can see examples of these connection strings in the SDK.

CrmServiceClient crmSvc = new CrmServiceClient(ConfigurationManager.ConnectionStrings["Xrm"].ConnectionString);

For Dynamics 365 online, the connection would be:

<connectionStrings>
    <add name="Xrm" connectionString="AuthType=Office365;Username=jsmith@contoso.onmicrosoft.com; Password=passcode;Url=https://contoso.crm.dynamics.com" />
</connectionStrings>

Thread Safety

The key to understanding performance and thread safety of calling the Organization Service is the difference between the client proxy and the WCF channel. As described by the 'Improve service channel allocation performance' topic from the best practice entry in the SDK, the channel should be reused because creating it involves time consuming metadata download and user authentication.

The old Microsoft.Xrm.Client was thread safe and would automatically reuse the WCF channel that was already authenticated. The Xrm.Tooling CrmServiceClient is no exception. You can create a new instance of CrmServiceClient and existing service channels will be reused if one is available on that thread. Any calls the same service channel will be locked to prevent thread issues.

To demonstrate this, I first used the following code that ensures that a single CrmServiceClient is created per thread.

Parallel.For(1, numberOfRequests,
    new ParallelOptions() { MaxDegreeOfParallelism = maxDop },
    () =>
    {
        // This is run for each thread
        var client = new CrmServiceClient(username,
               CrmServiceClient.MakeSecureString(password),
               "EMEA",
               orgname,
               useUniqueInstance: false,
               useSsl: false,
               isOffice365: true);
        
        return client;
    },
    (index, loopState, client) =>
    {
        // Make a large request that takes a bit of time
        QueryExpression accounts = new QueryExpression("account")
        {
            ColumnSet = new ColumnSet(true)
        };
        client.RetrieveMultiple(accounts);
        return client;
    },
    (client) =>
    {
    });

With a Degree of Parallelism of 4 (the number of threads that can be executing in parallel) and a request count of 200, there will be a single CrmServiceClient created for each thread and the fiddler trace looks like this:

Now to prove that the CrmServiceClient handles thread concurrency automatically, I moved the instantiation into loop so that every request would create a new client:

Parallel.For(1, numberOfRequests,
    new ParallelOptions() { MaxDegreeOfParallelism = maxDop },
    (index) =>
    {
        // This is run for every request
        var client = new CrmServiceClient(username,
               CrmServiceClient.MakeSecureString(password),
               "EMEA",
               orgname,
               useUniqueInstance: false,
               useSsl: false,
               isOffice365: true);
        // Make a large request that takes a bit of time
        QueryExpression accounts = new QueryExpression("account")
        {
            ColumnSet = new ColumnSet(true)
        };
        client.RetrieveMultiple(accounts);
    });

Running this still shows a very similar trace in fiddler:

This proves that the CrmServiceClient is caching the service channel and returning a pre-authenticated version per thread.

In contrast to this, if we set the useUniqueInstance property to true on the CrmServiceClient constructor, we get the following trace in fiddler:

So now each request is re-running the channel authentication for each query – far from optimal!

The nice thing about the Xrm.Tooling library is that it is used exclusively throughout the SDK – where the old Xrm.Client was an satellite library that came from the legacy ADX portal libraries.

Thanks to my friend and fellow MVP Guido Preite for nudging me to write this post!

@ScottDurow

Posted on 20. February 2017

How to get assistance without the Relationship Assistant!

Dynamics 365 has brought with it a new and amazing feature called the 'Relationship Assistant'. It is part of a preview feature (unsupported and US only) called 'Relationship Insights' which promises to bring some amazing productivity tools to the Dynamics 365 platform.

Relationship Assistant shows actionable cards in both the web client and mobile client using a mix of both plain old filter conditions and machine learning.

Machine Learning Cards

One of the most exciting part of the Relationship Assistant is the use of machine learning to examine the contents of your emails and predict what you need to do next:

Customer Question Card

Issue Detected Card

'Plain old query' Cards

Whilst the machine learning aspects may be out of our reach to us mere mortals at this time, the cards that are based on simpler filter conditions such as 'Due Today' and 'Meeting Today' are items that can be easily shown in a dashboard without this preview feature. Here are some examples of information that can be gained from simple date queries:

Due Today Card

Meeting Today Card

Missed Close Date Card

(Images taken from the Relationship Assistant Card reference - https://www.microsoft.com/en-us/dynamics/crm-customer-center/preview-feature-action-cards-reference.aspx)

Create your own 'Relationship Assistant' Dashboard

The main challenge with producing information shown above is the date aspect to the query. We can easily show a single set of records that use the 'Next X days' type of operator, but you could not easily use 'todays' date in a dashboard chart – at least not until CRM2015 introduced calculated fields. Now it is rather easy to produce a dashboard similar to the following:

  • The key feature of dashboards is that they are can be tailored to show your own data which can be drilled into to show the underlying records. This is comparable to the 'actionable' aspect of the relationship assistant where you could drill into the tasks due to 'today' and open them to work upon.

    Notice the field 'Due' that can have the value 'Completed', 'Due Next 7 Days', 'Due Today', 'Overdue', or 'Scheduled'. This field isn't stored as a persistent field in the database, but instead it is a calculated field so there are no nightly jobs or workflows required to update a field based on the current date.

    Adding a 'Due Status' field to an Activity Entity

    1. Create a solution with the Activity Entity that you want to add the 'Due Status' field to
    2. Create a new field called 'Due Diff' – this will give us a field that shows the number of days before/after the activity due date.
    3. Click 'Edit' and type the expression
      DiffInDays(scheduledstart, Now())
      Note: This assumes that this is an Appointment and you want to use the scheduledstart date to control the due date.
    4. Add a new global Option Set that holds the possible values for the Due status
    5. Create a new Calculated Option Set field called 'Due' on the Activity record. Use the Existing Option Set created above.
    6. Click 'Edit' on the Calculated Field type and add the following logic:
    7. Create a chart something like:
    8. Publish and add the charts to a dashboard!

    Of course other more complex options exist but with all the excitement and awesomeness of Machine Learning it is important to remember that we can achieve great things with just the right kind of queries, charts and dashboards!

    Hope this helps!

    Posted on 15. January 2017

    Smart Buttons in the Ribbon Workbench

    Happy 2017! This new year promises to be really exciting in the world of Dynamics 365. The spring release is going to be a big one for developers and I'm really looking forwards to it.

    In the meantime, I've released a new beta version of the Ribbon Workbench that includes the following features:

    1. Copy and Paste of Commands, Buttons & Enable/Disable Rules
    2. Free text JSON Clipboard – allowing you to see what is on the clipboard and potentially copy/paste between instances and even make text changes before pasting.
    3. Full support for Smart Buttons

    You can grab the latest beta of the Ribbon Workbench that support smart buttons from https://www.develop1.net/public/rwb/ribbonworkbench.aspx#DOWNLOAD

    Before I go into more detail about the JSON Clipboard, I wanted to tell you about Smart Buttons because I'm really excited about the possibilities that they may bring.

    So what are Smart Buttons?

    Smart Buttons are a concept I've been toying with since back in the Silverlight days of the Ribbon Workbench. They basically are buttons in the toolbox that add a predefined template of Ribbon Customisations that can be parameterised and contain references to prebuilt JavaScript web resources.

    When you add a standard button using the Ribbon Workbench you get to set the label, image and then reference a command that can in turn reference some JavaScript that you create separately.

    With a smart button, the JavaScript is already built for you and already installed in your Dynamics 365 organisation. Any solution that contains a smart button pre-build JavaScript must also contains a Smart Button manifest file that tells the Ribbon Workbench what's available and defines the templates it exposes.

    A typical uses of Smart Buttons are:

    Developers

    1. Install a Smart Button solution in Development Org (e.g. the Develop1 Smart Button Solution described later)
    2. Install Ribbon Workbench in Development Org
    3. Add Smart Button to Development Org using Ribbon Workbench
    4. Deploy Smart Button solution to Pre-Production/Production – The Ribbon Workbench would not need to be installed
    5. Deploy solution from Development (containing smart button configuration)

    Lead Developers

    1. Create a standard way of performing certain actions from Ribbon Buttons based on common library code they have created.
    2. Create a smart button manifest webresource and include that in their solution as well.
    3. Instruct Developers to use Smart buttons in the Ribbon Workbench rather than adding ribbon customisations manually.

    Third Party ISV's

    1. Create an ISV solution that allows functionality to be invoked in custom entities (I will be publishing a smart button manifest for my network view solution so you can add visualise buttons to any entity).
    2. Rather than publishing instructions on how to add buttons manually, create a smart button manifest that is picked up by the Ribbon Workbench
    3. When installed on the ISV customer's organisation, the Ribbon Workbench scans the organisation for any smart button manifest files and picks up the ISV's custom buttons that are available for use.

    I have built a simple Smart Button solution as a preview of this feature that provides the following:

    • Run Report – Allows you to select a report and create a short cut on any record form.
    • Run Workflow – Allows you to select a workflow and create a short cut to run it on a record form or sub-grid. You can also specify a call-back function to call when it is completed so that you can refresh the form or perform some other action. This is one of the most requested items on the Ribbon Workbench's uservoice site.
    • Run Dialog – This is similar to the Run Workflow button but starts a Dialog instead. Useful for creating your own custom 'Close Opportunity' forms.    
    • Quick JS – Sometimes you want to run some simple JavaScript (such as setting a drop down value and saving the form) when you click a command bar button. Quick JS allows you to simply create a button and directly specify the JavaScript without creating a separate Webresource to hold the code.


    You can download this preview Smart Button solution and give it a try from github: https://github.com/scottdurow/RibbonWorkbench/releases

    Creating your own Smart Buttons!

    In the next post I will describe the copy and paste JSON clipboard and how to use this to create a smart button manifest for your own solutions.

     

    Posted on 9. December 2016

    Dynamic365 Data Export Service

    If you've moved to Dynamics CRM/365 Online then the likelihood is that you've come up against the limitation of not being able to query the SQL database directly to perform more complex reporting or for custom integrations. Many on premises deployments rely on querying the backend databases and in the past this has been a blocker to moving to the cloud – or at least it has meant a complex and costly integration to copy the data from Dynamics 365 to a on prem SQL database.

    The introduction of the Data Export Service is a real game changer with the possibility to replicate your data from Dynamics CRM/365 online to an Azure SQL database in your own Azure Subscription. Once you have your data in a SQL Database you can then using PowerBI, integrate with other systems and create a data warehouse. I've found that the speed of the replication is impressive, being minutes/seconds and not hours.

    There are a number of perquisites to enabling this which you can read about in msdn: https://technet.microsoft.com/en-us/library/mt744592.aspx

    • Azure Active Directory linked to Office 365
    • Azure SQL Database and user with correct permissions
    • Azure KeyVault created (using PowerShell script provided)
    • Dynamics CRM Online 8.1 or later
    • Data Export Service solution installed from App Source
    • Change tracking enabled for custom entities you want to sync
    • You must be a System Administrator to create the export profiles

    The PowerShell script requires that you install the Azure cmdlets – see https://docs.microsoft.com/en-gb/powershell/azureps-cmdlets-docs/

    Here is a video that demonstrates this new service and how to set it up

    Posted on 21. November 2016

    Dynamics 365 Application User

    What is great to see with Dynamics 365 is the concept of the Application User. This means that subscribing systems can be connected to external systems using a 'Service Account' style user rather than the previous technique of providing username and passwords.

    One of the knock on effects of this is the new 'Application User' form on the User record. I've had some questions from people who see this form by default. You simply need to change the form back the standard 'User' form:

    If you try to save the user record on this form you will receive the error 'You must provide a value for Application ID' since this is a mandatory field.

    If you want to learn more about the Multi-tenant server to server application user integration scenarios that are used by App Source, then check out the following article:

    https://msdn.microsoft.com/en-us/library/mt790170.aspx

    Posted on 22. August 2016

    Will the Common Data Model change my Dynamics CRM Database?

    This is question on the lips of many after the recent Dynamics 365 anouncements.

    Watch my short video to see how Dynamics CRM sits alongside the Common Data Model: https://youtu.be/fYIPXx9zjj8

    Posted on 22. August 2016

    Debugging JavaScript in the Interactive Service Hub (Part 1)

    Those that read regularly my blog and follow my work with Sparkle XRM will know I'm a massive fan of using Fiddler to debug JavaScript. One of the most productive 'superpowers' that Fiddler gives us is the ability to change JavaScript on the disk and not have to upload/publish – we can simply refresh the form and the new script will be used.

    The Interactive Service Hub (ISH) was first introduced in CRM2016 and has been improved with more support for customisations in CRM2016 Update 1.

    I see the purpose of the ISH at this stage is not to replace the main User Interface but rather as a testing ground for the principle of bringing the MoCA mobile/tablet native client platform to the web client. I think of it similar to the introduction of the Polaris UI back in CRM2011 – there are many similarities in that they both only support a limited set of entities and have limited customisations features. The main difference is that the ISH is being incrementally improved with each release, where the Polaris UI was more of a throw away proof of concept. At this stage the ISH is only supporting 'case' oriented operations but I'm sure it'll eventually graduate to support all Sales, Service and Marketing features.

    So why the new approach the UI?

    Surely it would be better to improve the existing UI incrementally rather than replace it?

    One of the key drivers for the Dynamics CRM Team over the last few releases has been 'configure once deploy everywhere'. This allows us to configure business rules that can be run on all devices/platforms reliably without having to perform separate testing and perhaps re-write to target different clients. The maintenance of having multiple user interface platforms is considerable so it's a natural step to try and achieve some degree of convergence between the mobile/tablet/web/outlook interfaces.

    A little background on how the ISH loads metadata

    I think we are all fairly comfortable with the normal Web 2.0 paradigm of loading resources. This is where with each operation the client requests an html page and then the browser requests all the additional resources (JavaScript, CSS etc. ) that are referenced by that page. JavaScript can then make additional XHR/Ajax requests to the server to display further dynamic content. The CRM2016 UI is very similar on this front as can be seen below. I documented the CRM2013 script loading sequence which hasn't significantly changed even in CRM2016.

  • Page Load Sequence Diagram
  • Each time you open the web client, the homepage.aspx or Main.aspx has to request the metadata for the specific resource (view or form) and then combine it with the requested data. Although there is browser and server side caching in place, this is still costly in terms of the requests and rendering overhead of the browser. The 'turbo forms' update in CRM2015 Update 1 has really helped with the speed of this since it minimises the resources that requested with each navigation however fundamentally it is still limited by the page per browser request architecture.

    ISH works very differently…

    The ISH is more what we would call a 'single page application'. The sequence is very different in that there is an initial download of metadata and then subsequently all user interactions only request the actual data using the Organization.svc and OrganizationData.svc.

  • New Page Load Sequence Diagram
  • This single page approach has the advantage that it makes navigation super slick but with the rather annoying drawback that there is an initial wait each time the ISH is opened where the metadata changes are checked. The first time you open the ISH all the metadata is downloaded but from then on only the differences from the last open are downloaded. If there haven't been any changes then it's super quick because all the metadata is stored in the browsers indexed Database but if you've done a publish then the next open can take a while. Furthermore, the new metadata won't be downloaded until you close and re-open the ISH - this is different to the Web 2.0 UI and can lead to the client working with stale metadata for a time. The Indexed Database is one of the significant differences between and HTML5 single page app and a more traditional Web 2.0 architecture.

    Note: For now the ISH mostly uses the SOAP/Xml based Organization.svc rather than the new JSON based Web API.

    The speed of the metadata sync can be helped further by using the 'Prepare Client Customizations' button on the solution since this will pre-prepare the download package rather than waiting for the first person to open the ISH to detect the changes in the metadata. The difference between the MoCA client and the ISH is that the MoCA asks if the user wants to download the updates – presumably because you may be on a low bandwidth connection.

    So where does that leave us with respect to JavaScript debugging?

    If you've been keeping up so far (you have right?) then you'll realise that because the metadata (this includes JavaScript) will be all stored in the browser Indexed DB and not relying on the browser cache. As a result, we can't simply prevent the files from being cached and download the latest version with each page load as we used to do with Fiddler. We're back with the uncomfortable debug cycle of having to make a change to a JavaScript web resource, upload it to CRM, publish, close and re-start the ISH - urgh!

    To preserve our collective sanity, I've created a little debug utility solution that you can use to clear the cache of specific web resources so that you can quickly make changes to JavaScript on your local disk and then reload it in the ISH without doing a full publish cycle. Here is how:

    1. Install the latest build of SparkleXRM
    2. Install the Interactive Service Hub Debug Helper Solution
    3. Setup Fiddler's Auto Responder to point to your local webresource file as per my instructions.
    4. Start the ISH to load your JavaScript
    5. Make a local change to your JavaScript
    6. Open the ISH Debug Utility Solution configuration page and enter the name of your script name, then click 'Refresh JavaScript Webresource'
      Note: You can enter only part of the webresource name and it will use a regular expression to match.
    7. Use Ctrl-F5 on your ISH Page and when re-loaded the Web Resource will use the new version since the debug utility has forced a new download and updated the Indexed DB storage.

    Sweet – but what about the MoCA client?

    Obviously this technique is not going to work for mobile client running on an iPad, iPhone etc. The good news is that you can run the MoCA client in the Chrome browser in the same way you can run the ISH – just navigate to:

    <crmserver>/nga/main.htm?org=<orgname>&server=<crmserver>

    Note: You must be pre-authenticated for this to work.

    OnPrem

    http://dev03/nga/main.htm?org=Contoso&server=http://dev03/Contoso

    OnPrem IFD

    https://myorg.contoso.com/nga/main.htm?org=myorg&server=https:// myorg.contoso.com

    Online

    https://myorg.crm4.dynamics.com/nga/main.htm?org=myorg&server=https:// myorg.crm4.dynamics.com  

    Since the ISH and the MoCA client are build using the same platform you can now use the ISH Debug Helper from the same browser session to perform the same script refresh! This is actually an excellent way of testing out your Scripts on the MoCA client! For more information, check out the comments in this tip of the day.

    Looking forwards to the future

    I'd really encourage you to check out the ISH and use the New CRM Suggestions site to record anything you find that you would like to see in subsequent releases. Whilst I suspect that the existing 'refreshed' UI will be available for some releases to come, it is likely at some point to become the new 'legacy' UI and with on-going investment being made in the ISH style UI.

    In part 2 we'll look at some limitation of the ISH and how to get around them.

    Any comments, just tweet me! @ScottDurow

    Posted on 22. July 2016

    Install Project Service Automation (PSA) – Where’s it gone?

    If you have tried to install Project Service recently you might have found that it's disappeared from the 'preferred solution' list in the Office 365 admin portal. So where has it gone?!

    Although the CRM Online Help hasn't yet been updated to reflect the fact, it has now moved to the recently release and very exciting AppSource!

    On the 11th of July Michael Kushinsky of Microsoft helpfully posted in the dynamics community the new instructions on how to install and upgrade from a trial installation – and there will be an official blog post about it soon.

    I thought I would quickly show you how easy it is to use App Source to install PSA:

    1. Select Settings -> Dynamics Marketplace

    2. Search for 'Project Service' and click 'Try'

    3. Accept the T&Cs (you always read them in full right!)

    4. Wait for the solution to install and you're off!

    I can't wait to see AppSource grow and mature!

    Posted on 8. July 2016

    Mastering Ribbon Workbench 2016

    Since the release of the Ribbon Workbench 2016 I am in the process of updating the documentation to reflect the new user interface.

    This video series on mastering the Ribbon Workbench 2016 will take you through from installing to performing advanced customisations.

    Part 1 - Downloading & Installing

    Part 2 - User Interface Overview

    Part 3 - Hide Actions

    Part 4 - Moving Managed Buttons to a Flyout Menu

    Part 5 - Hiding buttons conditional to the form context

    More to follow!...

    Posted on 8. July 2016

    Ribbon Workbench Version Compatibility

    I'm pleased to announce that in addition to the managed solution that you can install inside Dynamics CRM, the Ribbon Workbench 2016 is also available in the XrmToolbox (if you hadn't already noticed!).

    When you open the XrmToolbox you will see that there is the Ribbon Workbench available for download in the plugin store.

    Keep checking out the store because tools are being added regularly by some great plugin authors.

    Thank you to all those who are helping to beta-test, I have been really encouraged by your comments and suggestions. The re-write of the Ribbon Workbench (to remove its dependency on Silverlight) and the XrmToolbox version has been on my 'to-do' list for much longer than I would have liked and so I'm particularly pleased with this release.

    The new Ribbon Workbench 2016 solution installs alongside the older version – so if you have an upgraded org you might get them all sitting there on your command bar. 

    I'm keeping there on one of my organisations for posterity but you can safely uninstall older versions without losing any of your customisations.

    Here is a version compatibility matrix for users of the older versions:

    Name

    Ribbon Workbench

    Ribbon Workbench 2013

    Ribbon Workbench 2016

    Icon

    Latest Version

    1.0.1.9

    2.0.1.3

    3.0.16

    Requires Silverlight?

    Yes

    Yes

     

    XrmToolbox Version?

       

    Yes

    CRM 2011
    (Inc. UR12+)

    Supported

       

    CRM 2013
    (Inc. SP1+)

     

    Supported

     

    CRM 2015
    (Inc. Update 1+)

     

    Supported

    Supported

    CRM 2016
    (Inc. Update 1+)

     

     

    Supported

    Supported