dazfuller
Documentation the easy way
So a slight departure from Spark (sort of) for this post, but I wanted to look at one of the most commonly overlooked aspects of building...
top of page
Pies, Lies and AIs
Showing the struggle of organisations to address the data revolution
Pies, Lies and AIs
Exploring the world of data and organisational intransigence
Search
dazfuller
Using Spark to read from Excel
People have data in Excel, so lets have a look at how we can read that data using Spark
dazfuller
Just one more column, what could go wrong?
Sometimes, when you go scanning through the documentation for Spark, you come across notes about certain functions. These tend to offer...
dazfuller
Why leave bad data to chance?
Something that we often see as Spark jobs are moved into production is that handling of bad data is either ignored, or a lot of effort...
dazfuller
Pivot, Step, Pivot, Twist, Un-pivot
Getting data into a good shape is a key part to Data Engineering, and we often get data in all sorts of shape and quality
dazfuller
Making development easier in Databricks
It's now easier to see the documentation directly in the notebook, rather than having to call "help" or go searching the Internet
dazfuller
Bringing Azure Databricks to Security Data
Back in October 2020 I had the amazing opportunity to with with Richard on a small project where we wanted to speed up processing of CEF...
azurecoder
Mapping source IP addresses to their countries of origin using Azure Databricks
A simple way to determine which country and IP address originated from at scale using Azure Databricks.
Home: Blog2
Home: Subscribe
CONTACT
Home: Contact
bottom of page