Setting Sub-Grid FilterXml In The Unified Interface And Other Naughty Things

This post started as a twitter poll where I asked if I should blog about an unsupported solution I developed for a rather unusual business requirement dealing with adding option set values to a control that didn’t actually exist in the Option Set of the system.  Because 5 more people voted for me to blog the solution than not, the code for that twitter poll is at the end of this blog post.  But, the real reason I suspect that most of you are here is to be able to set the fetchXml/filterXml of a sub-grid in the new Unified Interface of Dynamics CE / CDS, so let’s get started…

Most JS devs are capable enough to start snooping into the JS Dom of the grid control and find the setFilterXml function.  One would think calling this undocumented function would do what one desires, but nope, it does not.  There have even been attempts to re-write the function that may have worked at one time, but have never worked for me ( https://medium.com/@meelamri23/dynamically-set-fetchxml-to-subgrid-on-dynamics-365-v9-uci-a4a531200e73, https://community.dynamics.com/crm/f/microsoft-dynamics-crm-forum/299697/dynamics-365-unified-interface-inject-fetchxml-into-subgrid).  There is also the supported method of writing a plugin to edit the FetchXml on the server in a Retrieve Multiple plugin, (https://community.dynamics.com/crm/f/microsoft-dynamics-crm-forum/216881/how-to-set-up-custom-fetchxml-for-subgrid-in-dynamics-crm, https://sank8sinha.wordpress.com/2020/01/07/adding-filtered-views-in-uci-in-dynamics-365-crm-finally-achieved/) but unfortunately there is no client context and this may not be possible in some situations and requires a lot of extra effort.  

So what is the solution?  After hitting my head against the brick wall that is the setFilterXml function of the grid control, I decided to focus on figuring out how the fetch xml for the grid was getting determined in the first place.  When attempting to edit the metadata of an option set, as mentioned above, I discovered a function to access the global page state: “Xrm.Utility.getGlobalContext()._clientApiExecutor._store.getState()”.  This function returns a state object that contains the entire page model (metadata, ribbon rules, business process flows, etc).  I had used it to edit option set metadata to allow for dynamic values to be added to it (Picture a payment screen where customer’s previous payment information was in the option set drop down, which is way easier to select from rather than a lookup control.  The resultant function “resetOptions” is in the code block at the bottom of this page.) and I decided to see if I could find the metadata used to generate the Rest call to the server for the option set.

I fired up the Debugger window once more and dived into the call hierarchy used to generate the rest call to the server and discovered that the query was being built from the same metadata cache on the page.  “metadata.views” was an object with GUID properties of view metadata.  A couple quick edits in the debugger console and a refresh of the grid later, and I was in luck!  Editing the fetch xml in the metadata of the page state resulted in directly updating the fetch xml used to query and populate the grid results!  

I’ve since gone through and created a function to do the heavy lifting of finding the view id for the grid by the name of the grid control, and replacing any filters from the view with the filter xml provided as a parameter.  (Please note, this is unsupported.  It could break at any point so use it at your own risk.  With that being said everything I see points to that being unlikely for the foreseeable future)  The function is located below as TypeScript, because TypeScript is awesome and it’s not too hard to remove the typing if you’re just using plain old JS.  I’ve also gone through and documented the entirety of the GlobalState object returned from the “getState()” function as a TypeScript definition file, in the hopes that in the future I can use it to do more  “naughty” unsupported customizations.  You can access it here and is designed to go in your npm “node_modules/@types” folder.  (If someone wants to add it do the work of uploading it to GitHub and making it an npm package, be my guest!)

Call setSubgridFilterXml to set the sub-grid control fetch xml.  Since the metadata is shared at the page level, each grid will require a unique view to keep form interfering with other grids.

/**
 * Updates the Fetch XML of the Metadata which is used to generate the OData Query.
 * Since the metadata is shared at the page level, each grid will require a unique view to keep from interfering with other grids.
 * @param context Global Context
 * @param formContext Form Context
 * @param gridName Name of the Grid
 * @param filterXml Fetch Xml to set the Grid to
 */
export function setSubgridFilterXml(context: Xrm.GlobalContext, formContext: Xrm.FormContext, gridName: string, filterXml: string): void {
    console.info("Unsupported.setSubgridFilterXml(): Executing for grid: ", gridName, ", fetchXml: ", filterXml);
    const gridControl = formContext.getControl(gridName) as Xrm.Controls.GridControl;
    if (!gridControl) {
        console.warn(`No subgrid control found found name ${gridName} in Unsupported.setSubgridFilterXml()`);
        return;
    }
    try {
        const viewId = gridControl.getViewSelector().getCurrentView().id
            .toLowerCase()
            .replace("{", "")
            .replace("}", "");
        const view = getState(context).metadata.views[viewId];
        if (!view) {
            console.warn(`No view was found in the metadata for grid ${gridName} and viewId ${viewId}.`);
            return;
        }
        const originalXml = view.fetchXML;
        const fetchXml = removeFilters(removeLinkedEntities(originalXml));
        const insertAtIndex = fetchXml.lastIndexOf("</entity>");
        // Remove any white spaces between XML tags to ensure that different filters are compared the same when checking to refresh
        view.fetchXML = (fetchXml.substring(0, insertAtIndex) + filterXml + fetchXml.substring(insertAtIndex)).replace(/>\s+</g, "><");

        if (view.fetchXML !== originalXml) {
            // Refresh to load the new Fetch            
            gridControl.refresh();
        }

    } catch (err) {
        CommonLib.error(err);
        alert(`Error attempting unsupported method call setSubGridFetchXml for grid ${gridName}`);
    }
}

function getState(context: Xrm.GlobalContext) {
    return (context as XrmUnsupportedGlobalContext.Context)._clientApiExecutor._store.getState();
}

function removeFilters(fetchXml: string): string {
    return removeXmlNode(fetchXml, "filter");
}

function removeLinkedEntities(fetchXml: string) {
    return removeXmlNode(fetchXml, "link-entity");
}

function removeXmlNode(xml: string, nodeName: string) {
    // Remove Empty tags i.e. <example /> or <example a="b" />
    xml = xml.replace(new RegExp(`<[\s]*${nodeName}[^/>]*\\/>`, "gm"), "");

    const startTag = "<" + nodeName;
    const endTag = `</${nodeName}>`;
    let endIndex = xml.indexOf(endTag);

    // use first end Tag to do inner search
    while (endIndex >= 0) {
        endIndex += endTag.length;
        const startIndex = xml.substring(0, endIndex).lastIndexOf(startTag);
        xml = xml.substring(0, startIndex) + xml.substring(endIndex, xml.length);
        endIndex = xml.indexOf(endTag);
    }
    return xml;
}

This code can be used to allow for dynamically defining the option set values in an option set.  It will still fail on save if the integer value is not actually defined in the system

/**
 * Crm only supports filtering option sets.  This supports resetting the options, although it will still fail on save unless other precautions are taken.
 * It will allow for setting the value.
 * @param context Global Context
 * @param formContext Form Context
 * @param attributeName The name of the attribute
 * @param options The options to reset the Option Sets to
 */
export function resetOptions(context: Xrm.GlobalContext, formContext: Xrm.FormContext, attributeName: string, options: Xrm.OptionSetValue[]) {
    console.warn("Unsupported.resetOptions(): Executing for attribute: " + attributeName);
    const att = formContext.getAttribute(attributeName);
    if (!att) {
        console.warn(`No Attribute found for ${attributeName} in resetOptions.`);
        return;
    }

    const metadata = getState(context).metadata.attributes[att.getParent().getEntityName()][attributeName];
    const nonExistingValues = options.filter(v => {
        return metadata.OptionSet.findIndex(o => {
            return o.Value === v.value as any;
        }) < 0;
    }).map(osv => {
        return {
            Color: "#0000ff",
            DefaultStatus: undefined,
            InvariantName: undefined,
            Label: osv.text,
            ParentValues: undefined,
            State: undefined,
            TransitionData: null,
            Value: osv.value
        } as XrmUnsupportedGlobalContext.Metadata.OptionSetMetadata;
    });

    metadata.OptionSet = nonExistingValues.concat(metadata.OptionSet);
}

Happy Coding!

TypeScript Makes Writing Cleaner Code Easier

Clean Coders know that Boolean Arguments can be dangerous and as a general rule, should be avoided.  I had this thought in mind when writing a TypeScript unit test recently.  I had this bit of code which is rather ugly and not immediately obvious what it was doing.

getPaymentMethodsSpy.and.callFake(() => ActionsStubs.GetPaymentMethods.fakeResponseWithPaymentCount(1));
await Payment.setPayments("");
CommonLib.setValue(Payment.fields.paymentMethod, 1);

The fakeResponseWithPaymentCount sets the getPaymentMethodsSpy to generate the given number of fake existing payments for a count (1 in this case) as a response from the custom action, GetPaymentMethods.  Each odd numbered payment is a bank, and each even numbered one is a credit card.   So for this snippet, line 1 fakes a response of 1 Bank Payment, line 2 sets the Payment on the form, and line 3 selects that payment as the payment to use.  Simple right?  Not.  Good luck remembering this and re-creating it for processing a Credit Card.
So how does TypeScript here help make this cleaner?  Before we get there, lets do the obvious first step and refactor these statements out to a method:

async function selectExistingPayment(isBank: boolean) {
    getPaymentMethodsSpy.and.callFake(() => ActionsStubs.GetPaymentMethods.fakeResponseWithPaymentCount(2));
    await Payment.setPayments("");
    CommonLib.setValue(Payment.fields.paymentMethod, isBank ? 1 : 2);
}

This is way more usable, except when I’m looking at a calls site for it, it’s still really not clear:

selectExistingPayment(true);
selectExistingPayment(false);

What does false mean?  Don’t select it?  Sure, you could create an enum and define “credit card”, and “bank” as the two options, but this is just a unit test helper method.  I’m not exposing it anywhere, and even if I did, that’s a lot of extra work (Ok, so it’s a tiny bit of work, but it can feel like a lot #1stWorldProblems).  The simple solution TypeScript offers is string Union Types.

async function selectExistingPayment(type: "CC" | "BANK") {
    getPaymentMethodsSpy.and.callFake(() => ActionsStubs.GetPaymentMethods.fakeResponseWithPaymentCount(2));
    await Payment.setPayments("");
    CommonLib.setValue(Payment.fields.paymentMethod, type === "BANK" ? 1 : 2);
}

So now the calls sites look like this:

Extremely readable as to what I’m doing, and you get compile time error checking that "Dad" (or any other values besides "CC" and "BANK" ) is not a valid Existing Payment Method!

Making CrmWebApi Entity References Suck Less For Creates and Updates

For those that “grew up” on the 2011 Rest endpoint for CRM, attempting to populate Entity Reference attributes for create or update calls feels rather painful in the new CrmWebApi.

account["primarycontactid@odata.bind"] = "/contacts(E15C03BA-10EC-E511-80E2-C4346BAD87C8)";

Was it “"@odata.bind” or “@bind.odata”? Was it a forward slash or backward slash?  Did the Guid have curly braces?

Yes it’s a small pain, but it is bigger if you normally use field accessors (“entity.field” rather than array accessors: “entity[‘field’]”) because "account.primarycontactid@odata.bind" isn't a valid field name.  It’s probably because of my C# background, but I prefer not to use the object array accessor method when possible.  So the question is, how to make this syntax better and help me remember it.

On my current project I use David Yack’s CRMWebAPI.  It’s simple, and uses standard Promises, so no need for a new library, just polyfill Promises (if you’re using IE 11) and you’re all set.  The calls are wrapped by a custom TypeScript library (CrmWebApiLib) to allow for some custom changes, of which, this implementation is one.  First, the library defines an Entity Reference class (*Note, this is TypeScript, get it, use it, love it)

export class EntityReference implements ODataFormattable {
    constructor(public collectionName: string, public id: string) { }
 
    toODataFormat = (): string => {

        return `/${this.collectionName}(${CrmWebApiLib.removeCurlyBraces(this.id)})`;
    }
 
    getODataPropertyName = (propertyName: string): string => {
        return `${propertyName}@odata.bind`;
    }
}

The class has two public properties, “collectionName” and “id”, and implements the two functions of the ODataFormattable interface, “toODataFromat” and “getODataPropertyName”.  The toODataFormat adds the forward slash and formats the guid correctly, and the getODataPropertyName appends the “@data.bind” to the property name parameter.

The ODataFormattable interface just defines the two functions.  Then there is also a User Defined Type Guard to determine if any given object implements the ODataFormattable interface:

export interface ODataFormattable {
    toODataFormat(): string;
    getODataPropertyName(propertyName: string): string;
}

export function isODataFormattable(arg: any): arg is ODataFormattable {
    const formattable = arg as ODataFormattable;
    return formattable && formattable.toODataFormat !== undefined && formattable.getODataPropertyName !== undefined;
}

This then is all used in the prepareForOData function:

/**
 * Loops through properties, searching for any ODataFormattable properties or arrays with ODataFormattable, and updates the format to be OData Friendly
 * @param data
 */
function prepareForOData(data: any): any {
    const oData = {};
    for (const propName in data) {
        if (!data.hasOwnProperty(propName)) {
            continue;
        }
 
        const value = data[propName];
        if (isODataFormattable(value)) {
            oData[value.getODataPropertyName(propName)] = value.toODataFormat();
        } else if (value instanceof Array) {
            oData[propName] = value.map(prepareForOData);
        } else {
            oData[propName] = value;
        }
    }
    return oData;
}

It creates a new object, and basically loops through all properties of the data object, copying it over to the new object.  If the value of the property is a ODataFormatable, it will update the value as well as the property name.  There is then a recursive map call to handle arrays as well (think party lists).  prepareForOData is then called from within the create and update methods:

export function create(entityCollection: string, data: any): Promise<any> {
    return instance().Create(entityCollection, prepareForOData(data));
}
 
export function update(entityCollection: string, key: string, data: any, upsert?: boolean): Promise<any> {
    if (key.indexOf("{") >= 0 || key.indexOf("}") >= 0) {
        key = CrmWebApiLib.removeCurlyBraces(key);
    }

    return instance().Update(entityCollection, key, prepareForOData(data), upsert);
}

And now, these two calls, will result in the same exact request made to the CrmWebApi:

No Bueno

const note = {};
note["notetext"] = CommonLib.getValue(fields.description);
note["objectid_allgnt_location@odata.bind"] = `/allgnt_locations(${CommonLib.getSelectedLookupId(fields.location)})`;
CrmWebApiLib.create("annotations", note);

Muy Bueno

const note = {
    notetext: CommonLib.getValue(fields.description),
    objectid_allgent_location: new CrmWebApiLib.EntityReference("allgnt_locations", CommonLib.getSelectedLookupId(fields.location))
};
CrmWebApiLib.create("annotations", note);

How To Add Custom Filters That Are Asynchronous In Dynamics 365 Customer Engagement

Adding custom filters to lookup entities isn’t entirely simple.  It involves adding a trigger to the PreSearch function in which a call is made to set the filter on the control using addCustomFilter. The PreSearch event must be synchronous since it is triggered once the user has selected the lookup, and needs to apply the filter right then. 

But what if you want to do something like query the lookup entity for the given filter, and relax the criteria if nothing is found (i.e. Search by First and Last name, unless no match is found, then search by Last Name only).  That either involves a blocking synchronous call (Yuck), or some form of black magic to make something asynchronous behave synchronously.  Well, maybe not black magic, just 40 lines of TypeScript:

const _preSearchFilters = {} as { [index:string]:string };
 
/**
 * Handles adding a PreSearch that is Asynchronous.  To be called in the onLoad of the form.  This will trigger the getFilter method to attempt to assign the filter.
 * The Lookup Attribute will be disabled until the getFilter method has finished executing
 * @param info Object that contains the following properties:
 *             control - Name of the lookup control to add the PreSearch Filter for.
 *             filteringEntity - If entityLogicalName is not specified, the filter will be applied to all entities valid for the Lookup control.
 *             getFilter - Function that returns the Promise of the filter Xml.
 *             triggeringAttributes - List of attributes, that if changed, will result in the filter needing to be updated.

 */
export function addAsyncPreSearch(info: { control: string, filteringEntity?: string, getFilter: () => Promise<string>, triggeringAttributes?: string[]}) {
    const setAsyncFilter = async () => {
        const enablePostFilter = !Xrm.Page.getControl(info.control).getDisabled();
        if (enablePostFilter) {
            Xrm.Page.getControl(info.control).setDisabled(true);
        }
 
        try {
            _preSearchFilters[info.control] = await info.getFilter();
        } catch (e) {
            console.error(`Error occurred attempting to get the preSearch filter for ${info.control}`, e);
            _preSearchFilters[info.control] = "";
        } finally {
            if (enablePostFilter) {
                Xrm.Page.getControl(info.control).setDisabled(false);
            }
        }
    };
 
    Xrm.Page.getControl(info.control).addPreSearch((context: Xrm.Page.EventContext) => {
        const ctrl = (context.getEventSource() as Xrm.Page.LookupControl);
        if (ctrl && ctrl.addCustomFilter) {
            ctrl.addCustomFilter(_preSearchFilters[ctrl.getName()], info.filteringEntity);
        }
    });
 
    if (info.triggeringAttributes && info.triggeringAttributes.length > 0) {
        for (const att of info.triggeringAttributes) {
            Xrm.Page.getAttribute(att).addOnChange(setAsyncFilter);
        }
    }
 
    setAsyncFilter();
}

That’s a lot of code, let’s walk through it.

First, a module level field (_preSearchFilters) that stores the filters for each control is declared and instantiated (note there is an assumption that you will only ever have one filter per control, which I think is pretty safe).  For those of you new to TypeScript, “{ [index:string]:string }” is how you define that the field is an object, which is indexable by a string, returning string.  This would be equivalent to a C# Dictionary<string,string>.

Next is a nested function “setAsyncFilter” that wraps the async function “getFilter” that is passed in.  It handles 2 things, disabling the control until the async function finishes, and storing the result of the async function in the “_preSearchFilters”.  If the control doesn’t get disabled, then there is a chance that the user could attempt to perform a search the async function determines what the filter should actually be.

After the “setAsyncFilter” definition comes the first code that is actually executed, a call to get the control, and add an anonymous function as a preSearch.  The function just calls “addCustomFilter” on the control that triggered the action, with the optional filteringEntity parameter.

Next to last is an if statement to trigger the “setAsyncFilter” function, each and every time a field is updated that could potentially change the filter.  This if followed by the last step, a call to “setAsyncFilter” to initialize the value in the “_preSearchFilters: field.

To use this function, just call it in the onLoad of the form:

CommonLib.addAsyncPreSearch({
    control: Lead.fields.installFee,
    filteringEntity: "new_installationfee",
    getFilter: getInstallationFilter,
    triggeringAttributes: [Lead.fields.campaign,
                           Lead.fields.callType]
});

The call above adds an Async PreSearch to the installFee control.  The filter is to be applied to the “new_installationfee” entity, and is generated by the Promise returning function, “getInstallationFilter”.  Finally, a trigger is added to refresh the filter whenever the campaign or callType fields are updated.  That's it.

Chalk one up to TypeScript for handling the async/await.  Just don’t forget to polyfill your Promise call, or your IE 11 users won’t be too happy