Histograms, Bar Charts and Heatmaps for Feature Types

1.Why visualize features before modelling

Exploratory data analysis (EDA) is where you learn what a dataset looks like before you choose how to prepare it or which model to use. Databricks describes EDA as analyzing and visualizing data for four purposes: to find its main characteristics, spot patterns and trends, detect anomalies, and understand how variables relate to each other. The last purpose matters most for an ML associate, because the docs note that what you learn here can affect which algorithms you train.

Summary numbers alone hide a lot. Two features can have the same mean but completely different shapes, and an average can hide an outlier. Charts reveal patterns that numbers alone can miss. A Databricks notebook gives you two ways to make them. The first is the built-in visualization editor, which you can open on any display() result. The second is a Python library such as Plotly or Matplotlib. The EDA tutorial uses both.

Sources12

2.Continuous features: the histogram

A continuous feature, such as a price, a duration or an energy consumption figure, can take any value in a range, so counting each separate value tells you little. A histogram instead groups values into ranges and plots how often each range occurs. Databricks says a histogram shows whether values are clustered in a few ranges or spread out. It is drawn as a bar chart where you set the number of bars, called bins.

The bin count is the main setting to adjust. With too few bins, a distribution with two peaks flattens into one block. With too many, it turns into noise. The documentation's example plots o_totalprice from samples.tpch.orders with 20 bins.

The X axis also has a Scale option. Some features have values that span several orders of magnitude, so most of the data gets squashed into the first bar. Setting the scale to Logarithmic spreads those values out so you can see the shape. Histograms also support backend aggregation, so the chart covers more than the first 64K rows instead of a cut-off sample.

Sources3

3.Categorical features: bar and pie charts

A categorical feature, such as region, order priority or return flag, has a fixed set of levels. The useful question is not how the values are distributed, but how some metric compares across the levels. A bar chart answers that question. It draws one bar per category, and each bar's height is an aggregate such as a sum or a count. Databricks describes bar charts as showing change over time or proportionality, like a pie chart.

The EDA tutorial does this with Plotly. It filters the data to six regions, totals the emissions for each region with groupby(...).sum(), and passes the resulting Series to px.bar.

# Calculate total emissions for each region
regional_emissions = df[df['country'].isin(regions)].groupby('country')['greenhouse_gas_emissions'].sum()

# Plot the comparison
fig = px.bar(regional_emissions, title="Greenhouse Gas Emissions by Region")
fig.show()

The tutorial reads a ranking from this chart: Asia is highest, and Oceania, South America and Africa are lowest. A pie chart shows the same kind of proportions between categories, but the docs add a limit: pie charts are not meant for time series data. If you build either chart in the Databricks editor, string columns can only be aggregated with Count or Count Distinct. That is exactly what you need when each bar should show how often its category occurs.

# Plot the comparison
fig = px. ? (regional_emissions, title="Greenhouse Gas Emissions by Region")
fig.show()

Sources34

4.Two categorical features at once: heatmaps and pivot tables

You often want to see how two categorical features interact, for example whether order status depends on order priority. A heatmap puts one category on each axis and colours each cell by a number. In the Databricks example, o_orderpriority is the X column, o_orderstatus is the Y column, and the colour shows the average of o_totalprice. Warmer colours usually mark the highest values and cooler colours the lowest.

A pivot table visualization shows the same cross-tabulation as numbers. Its rows come from one column (l_returnflag) and its columns from another (l_shipmode). Each cell holds a summed value and can be coloured by that value. It works like a SQL PIVOT or GROUP BY.

Sources3

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.A pie chart is a fine way to show how category shares change from year to year.Why is that wrong?

   Pie charts show proportions between metrics at a single point. Databricks states they are not meant for time series data, so use a bar or line chart instead.

   Covered in Categorical features: bar and pie charts

2. 2.A histogram is just a bar chart of a categorical column.Why is that wrong?

   A histogram plots how often values occur in a column, usually a continuous one, by grouping them into bins whose number you set. A bar chart compares an aggregate across categories.

   Covered in Continuous features: the histogram