Pies, Lies and AIs

Well the number of people looking for work in the cloud and data sector has now shot up as everyone adjusts their forecast expectations...

The nature of Elastacloud is beginning to change. Most of my advisors and sales team have never written a line of code in their life and...

I was stuck in traffic today for a few hours on the way to New Scientist Live. Missed loads of it. My bad planning. Worst luck. Really...

I had a great time last weekend with a garden party with my friend and one of my longstanding tech leads David. It was great to spend the...

I've taken the last week off. For the first time in about a decade I haven't responded to emails (at least I've picked and chosen), Teams...

I was mulling over whether to derive a new jocular term for a Data Mesh. I pondered Data Mess but that seemed to obvious so I've opted...

So since writing the original post about how to drop a SQL table from a Spark Notebook I've been meaning to follow it up with a version...

For the Python version of the code below, see the follow-up post. One of the nice things with Spark Pools in Azure Synapse Analytics is...

Note: this post is mainly about Azure but it can apply to any cloud. You get to a stage when you hear enough definitions of what people...

Sometimes, when you go scanning through the documentation for Spark, you come across notes about certain functions. These tend to offer...

Something that we often see as Spark jobs are moved into production is that handling of bad data is either ignored, or a lot of effort...

The command line is a powerful environment that lets you do a lot of work quickly, easily, and in a repeatable way