Pies, Lies and AIs

Exploring the world of data and organisational intransigence

dazfuller

Aug 26, 20216 min read

Documentation the easy way

So a slight departure from Spark (sort of) for this post, but I wanted to look at one of the most commonly overlooked aspects of building...

dazfuller

Jul 3, 20214 min read

People have data in Excel, so lets have a look at how we can read that data using Spark

dazfuller

May 22, 20213 min read

Sometimes, when you go scanning through the documentation for Spark, you come across notes about certain functions. These tend to offer...

dazfuller

Apr 25, 20216 min read

Something that we often see as Spark jobs are moved into production is that handling of bad data is either ignored, or a lot of effort...

dazfuller

Feb 21, 20213 min read

Getting data into a good shape is a key part to Data Engineering, and we often get data in all sorts of shape and quality

dazfuller

Feb 6, 20212 min read

It's now easier to see the documentation directly in the notebook, rather than having to call "help" or go searching the Internet

dazfuller

Jan 31, 20215 min read

Back in October 2020 I had the amazing opportunity to with with Richard on a small project where we wanted to speed up processing of CEF...

azurecoder

Dec 30, 20203 min read

A simple way to determine which country and IP address originated from at scale using Azure Databricks.

Home: Blog2

Home: Subscribe

Home: Contact