Dynamics Field Service loves Bing but what if you are in this situation?

Recently I had a situation in my project where I had to integrate with Google Map API instead of Bing Spatial Data Service (Or in simple works Bing Maps) in Field Service Scheduling Board. This post covers topics of why did I have to choose Google Maps API over native Bing Spatial Data Service, how does it work and some lessons learned.

Field Service Scheduling service comes with native, Out-of-the-box integration with Bing Spatial Data Service. There are actions in the Dynamics platform which lead to calling Bing Spatial Data Service. These two actions are msdyn_GeocodeAddress and msdyn_RetrieveDistanceMatrix actions.

*** I strongly suggest to use Bing over google when it comes to Dynamics Field Service, it is native and there is no and it works always better! The below case is a unique case and you may have a difference experience. I would love to hear from you about your experience but what we faced and solved was something which we could not find any other solution for ***

1. msdyn_GeocodeAddress is called to obtain coordinates (Longitude and Latitude) of an address by providing descriptive address. In simpler words we will pass address street, address city and address post code and in turn we get two decimal values presenting the physical address. This action is being called on save of any GeoLocation enabled entity. This entity will take address information and return longitude and latitude fields. Therefore, we can see this action triggering on Account and Contact entities if we enable Geocoding.
    
2. msdyn_RetrieveDistanceMatrix is called to obtain list of all coordinates near to a specific coordinate. Since coordinates are numeric translations of addresses, we can say this action retrieves all addresses with distance and travel time around a specific address.

In context of Field Service, our resources need locations information in order to be identified by the scheduling board and thus the platform calls msdyn_GeocodeAddress on Account and Contact creation and Update to ensure resources have the most updated coordinates. And on load of the schedule board, the platform calls msdyn_RetreiveDistanceMatrix by passing the coordinates of the work order or any other entity which we have enabled it for use in the schedule board. The msdyn_RetreiveDistanceMatrix then returns list of all resources with distance and estimated time of arrival.

The challenge

Our client was using a legacy system which was integrated with Google Map API. The legacy system was performing exactly the same functionality we expect from the schedule board. However the problem was that results on the schedule board was different than results on their legacy system. Till now everything makes sense because we know their legacy system was using Google Map API and our solution was using Bing Spatial Data Service. However there was a big problem in our solution which was impacting its acceptance and adoption because the result of Google Map API was more accurate than our solution. The first challenge was that the distance shown on the schedule board was empty or different than the legacy system for the same set of test data. The second challenge was that the arrival time and traffic data was not also as accurate as Google Map API.

The Solution

We started by analyzing to see why we are getting a different results. We took several steps which I am listing down here with the hope that it will help someone in future.

1. We started with analyzing resources on the schedule board where the distance was empty and the arrival time was also empty. We identified that our NOT ALL of our accounts, contacts and resources have longitude and latitude. It was a surprise for us because our entities have address by why the longitude and latitude was empty? Analyzing it further we noticed that these identified entities were missing country field in their address. From end user perspective it is obvious that all Accounts and Contacts and Resources are based in Australia so they were not adding Country to the addresses. However, the platform needs a proper address and a clean set of data. The msdyn_GeocodeAddress action needs a properly formatted address of Street1, (optional) street2, (optional) street3, Post Code, City and Country fields. Without these fields it will not be able to calculate a valid coordinates. By adding the country field to the address fields, we were able to fix this issue.
    
2. The next task was convincing the client to be happy with Bing Map out of box results due to its performance and native nature. However, the client is doing a critical health care service and showing 1 minute deviation in 1 hour travel could be acceptable; however showing 8 minutes difference in 14 minute drive was certainly not acceptable. We tried hard to understand why this difference in result is happening. We opened a case with Microsoft support but the answer we received was simple and unacceptable. They responded to us by saying "The platform passes geolocation information to Bing Spatial Data Service and only consumes the result obtained by the Bing Spatial Data Service. The Dynamics platform has no influence over the service and they cannot help us".
   I tried to run fiddler and see what is being passed from the platform and I reached to the same conclusion that the platform was just calling a webservice and the results were fixed. This was a big upset for us because I really did not want to go custom way since it adds complications to our solution and also the client had to pay for Google Map API service as well.
    
3. At last we reached to a conclusion that due to critical nature of the business we needed a more accurate solution and we have to move by getting Google Map API service. We had to write two actions in Dynamics Platform and use Google Map API instead of out of the box solution. Once we overwrote the actions, the platform was calling our code instead of the existing code and we could get the result which was much more acceptable to the business. The sample source code of these actions is found here and there is also a sample project here.

Lessons Learned

1. The inaccuracy of distance and arrival time on the schedule board may be due to your data. Examine and clean your data to reduce chances of error.
    
2. Before start of development we analyzed whether we need to override msdyn_GeocodeAddress or msdyn_RetrieveDistanceMatrix or both. What we identified is the Bing Spatial Data Service is not very accurate in identifying geolocation details for some addresses whereas the Google Map API was able to give us accurate geolocation information for the same address. An additional problem was that the Bing Spatial Data Service was not giving us the exact location for large blocks like hospitals or parks. We were expecting to get at least one coordinate for a hospital or park in order to calculate the arrival time but in such cases the Bing Spatial Data Service simply does not return anything which was not helping us.
    
3. Another issue was with getting the arrival time and using traffic information. We had issues varying between 5% to 30% difference in travel time for long distances and for short distance the difference was reaching to nearly 50% and this was not acceptable for business to operate. Therefore we decided to implement both actions in our solution.
    
4. msdy_RetrieveDistanceMatrix action calls the Google API service 3 times. In each turn it changes the inputParameters sequence to get an accurate result so you should leave the solution to call the API 3 times. However, calling the API 3 times will have a big impact on your pocket. More on this on the point 7.
    
5. There is no harm in convincing business to use out of the box schedule board with some difference in distance or arrival time.
    
6. Google Map API service is not free. Couple of years back, Google used to issue a trial key for accessing its Google Map API but the free trial version is there anymore. You will need to create a Billing Account (even for trails) and then use the Google Developers website to obtain the trial key. You will need this key in your actions and code.
    
7. Google Map API for msdyn_GeocodeAddress is quick cheap. However, the msdyn_RetrieveDistanceMatrix is a premium service and you will end up paying 25 cents for each API call and in total you will end up paying 75 cents for one time msdyn_RetrieveDistanceMatrix action. You will need to do a cost analysis before going with this approach. However, you can reduce the cost if you send less location to calculate the distance from. For example, if you want to see distance of all resources in 100 miles of your Account, the schedule board will send all resources in nearby location and it can be many but if you send limited number of resources (say 20) to check the distance, it will be cheaper.
    
8. The sample mentioned above, does not handles errors. You will end up receiving responses like "MAX REQUEST EXCEED" if your distance matrix is more than 625. So you will need to wait for a minute before your next request. There are more errors which you will need to take care and all of the response types are documented in the google map API page.

I hope this helps.