Pies, Lies and AIs

Exploring the world of data and organisational intransigence

azurecoder

Jul 23, 20244 min read

Richard vs the Microsoft Speech SDK round 2

When the Python Speech SDK fails on Linux

dazfuller

May 1, 20222 min read

So since writing the original post about how to drop a SQL table from a Spark Notebook I've been meaning to follow it up with a version...

dazfuller

Oct 21, 20214 min read

For the Python version of the code below, see the follow-up post. One of the nice things with Spark Pools in Azure Synapse Analytics is...

dazfuller

Aug 26, 20216 min read

So a slight departure from Spark (sort of) for this post, but I wanted to look at one of the most commonly overlooked aspects of building...

dazfuller

Jul 24, 20219 min read

Having recently released the Excel data source for Spark 3, I wanted to follow up with a "lets use it to process some Excel data" post....

dazfuller

Jul 3, 20214 min read

People have data in Excel, so lets have a look at how we can read that data using Spark

dazfuller

May 22, 20213 min read

Sometimes, when you go scanning through the documentation for Spark, you come across notes about certain functions. These tend to offer...

dazfuller

Apr 25, 20216 min read

Something that we often see as Spark jobs are moved into production is that handling of bad data is either ignored, or a lot of effort...

dazfuller

Feb 21, 20213 min read

Getting data into a good shape is a key part to Data Engineering, and we often get data in all sorts of shape and quality

dazfuller

Feb 15, 20216 min read

The command line is a powerful environment that lets you do a lot of work quickly, easily, and in a repeatable way

Home: Blog2

Home: Subscribe

Home: Contact