Geolocation in Fabric Real Time Intelligence

Lots of announcements at Build 2024. It's easy to miss the significant ones in lieu of the noise of the multitude of stuff.

I've been using Real Time Hub for a while now in Fabric. It's a boon for people that are not me and Elastacloud and need basic features to ingest Streams into Fabric. It's far from complete over the complex building blocks of Spark Streaming and Stream Analytics but it does have some powerful abstractions which make real time less scary.

I'm going to spend some time going through a few posts which will highlight some of these updates and their relevance. For now I'm struggling typing as the cat successfully swiped at my finger and every right-hand tap is a killer.

We recently built out two real-time solutions around fleet management which allow data from lots of vehicles to stream into Fabric via EventStreams and then in Kusto. One of the things that becomes very possible is to build out a real-time filter on a the Kusto Dashboard map.

Kusto Dashboard maps can be a little flaky. There are some caching issues which put the vehicle fleet off the coast of Somalia on the odd occasion. A key here is to ensure that the polygons aren't too complicated. We can build these ourselves though. In this instance I did this using geojson.io which allows us to build out polygons on the map and then export them as GeoJSON. The image shows how to build out polygons with fewer edges than you would find in a shape file.

What this means is that you can build out Kusto functions to ensure that you can filter regions or areas on a map.

The following KQL function takes the GeoJSON you can see in the image and allow lats and longs to be passed into the function to filter down on whether a point is in Scotland. This is done through the geo_point_in_polygon inbuilt function.

.create function IsInScotlandRegion(latitude:real, longitude:real)
{
    let GreaterScotlandPolygon = dynamic({
    "type": "Polygon",
    "coordinates": [
          [
            [
              -2.082081594279913,
              55.8157714841559
            ],
            [
              -0.22550176287018076,
              60.997982605241475
            ],
            [
              -1.3863838168968527,
              61.06990266611555
            ],
            [
              -9.78654990948121,
              57.951587901512
            ],
            [
              -6.8996400476373765,
              55.85102604688106
            ],
            [
              -4.851855779088027,
              54.51361966314133
            ],
            [
              -3.7140797989307544,
              54.58587400300186
            ],
            [
              -2.2771484554597237,
              55.507485850078666
            ],
            [
              -2.3509185441140232,
              55.65595738167232
            ],
            [
              -2.082081594279913,
              55.8157714841559
            ]
          ]
        ]
    });
    geo_point_in_polygon(latitude, longitude, GreaterScotlandPolygon)
}

We should just then be able to add a dropdown. Sorry for the crappy graphic. I'm still getting used to a mac. Using Seashore for blurs. Not feeling it though.

The beautiful thing about this now is that you can then just build out filters into the map to check all the points in Scotland. Grrr, mac, so rubbish. Will repost with proper readable Gaussian filters for now you get the gist.

Easy thing then is to build out a query linking the filter dropdown and the map together via the regional functions you've created. In this query the first where clause filters what the map will show. We had outliers in ours which gave us an extended map. You can either get rid of those from the dataset or from the mapset. I chose the latter. This focuses us on the UK. The second simply ensures that the point gets labelled in the

| where latitude >= 50.0 and latitude <= 58 and longitude >= -10.0 and longitude <= 2.0 
| extend AllRegions = case(IsInEastAngliaRegion(longitude, latitude), "East Anglia",                         IsInEastMidlandsRegion(longitude, latitude), "East Midlands",                         IsInLondonRegion(longitude, latitude), "Greater London",                         IsInNorthWestRegion(longitude, latitude), "North West",                         IsInNorthEastRegion(longitude, latitude), "North East",                         IsInNorthernIrelandRegion(longitude, latitude), "Northern Ireland",                          IsInScotlandRegion(longitude, latitude), "Scotland",                          IsInSouthEastRegion(longitude, latitude), "South East",                         IsInSouthWestRegion(longitude, latitude), "South West",                         IsInWalesRegion(longitude, latitude), "Wales",                         IsInYorkshireAndHumberRegion(longitude, latitude), "Yorkshire and Humber",                         "Other")

Real Time Intelligence will be a panacea for those wanting to bring real-time behaviours into Fabric. You still need to be innovative though. As we go through our journey and do the piece of innovation like this I'll push it back out for comment and reflection.

For now I'll nurse my finger and whimper a little. I noticed that I haven't blogged for about a year. Generally for the last 12 months I've been miserable but I'm back on form and enjoying solving problems for our customers so keep reading my musings if you want to hear all about it.

BTW, thanks to Amanda and Darren for their amazing contributions over the past few weeks. Happy trails.