5.5: Multivariate Analysis Techniques

Multivariate Analysis Techniques

Multivariate analysis in the context of EDA refers to examining relationships among three or more variables simultaneously. While humans can visualize 2D (and somewhat 3D) plots readily, higher dimensions require more advanced approaches. In practice, multivariate EDA often means either layering or faceting simpler plots by additional variables, or using techniques to reduce dimensionality and visualize in 2D.

Figure \(\PageIndex{1}\): Multivariate EDA Techniques Overview. This infographic summarizes the main strategies for exploring relationships among three or more variables, from visual encoding methods to dimensionality reduction. (CC BY 4.0; Felix Amoruwa) Access to a detailed description of Figure \(\PageIndex{1}\)

Visualization of Multiple Variables:

One straightforward way to incorporate a third variable is by using color, shape, or size encoding in a plot. For example, in a scatter plot of X vs Y, points could be colored by a third categorical variable (group), or sized by a third numeric variable. Alternatively, one can use faceted plots (also known as trellis or lattice plotting): create a grid of plots, each representing a subset of the data. For instance, a scatter plot of tip vs bill could be drawn separately for smokers and non-smokers (two panels), or even four panels for smoker vs non-smoker and male vs female combinations. The example of tips vs bill by gender and smoking that we described effectively does this in one combined plot (color and shape were used to distinguish groups. These methods allow the analyst to see interactions: e.g., maybe a trend holds in one group but not another, or two variables correlate overall but that is driven by a third factor.

Another visualization for multivariate numeric data is the scatter plot matrix mentioned earlier. If we have, say, 4 numerical variables, a 4×4 grid of scatterplots (each vs each) can be inspected to see pairwise relations. On the diagonal, you might put histograms of each variable. Such a matrix gives a comprehensive view of any linear or simple nonlinear relationships among all pairs, and one can sometimes notice clusters of points that persist across several plots (hinting at latent groupings).

Dimensionality Reduction:

When dealing with high-dimensional data (many variables), EDA often employs dimensionality reduction techniques to project the data into a lower dimension (usually 2D for plotting) while preserving as much structure as possible. One widely used method is Principal Component Analysis (PCA). PCA finds new orthogonal variables (principal components) that capture the most variance in the data. By plotting the first two principal components, one can often visualize the broad structure of high-dimensional data – for example, seeing if observations cluster or if there are outliers far from the cluster in PC space. PCA is extremely useful in EDA for tasks like identifying groups in data (perhaps corresponding to different types of observations) or checking if one or two dimensions explain most variability (which might justify simplifying the dataset). Other dimensionality reduction methods include multidimensional scaling (MDS), t-SNE, UMAP, and more, each with their own strengths in revealing structure (like clusters or manifold structure) in multivariate data.

Clustering and Grouping:

Another angle for multivariate EDA is to perform unsupervised learning techniques such as clustering (e.g., K-means or hierarchical clustering) to see if the data naturally groups into clusters based on multiple variables. Applying clustering during EDA can highlight whether there are distinct subpopulations in the dataset. For instance, if you have a dataset of customers with multiple attributes, a clustering algorithm might suggest there are, say, 3 types of customers. While clustering results themselves might be part of a modeling step, using them in EDA can be exploratory – you’re not yet validating that those clusters are “real” in a statistical sense, just seeing if the pattern emerges. If it does, you might then label those clusters and examine them separately, leading to richer insights.

Example – Correlation Heatmap: Suppose we have a dataset with 8 financial indicators for a set of companies. We might create a correlation heatmap to inspect all pairwise correlations at once. In such a heatmap, highly correlated pairs light up (for example, revenue and profit might show a strong positive correlation in bright color). This could immediately tell us if some variables are redundant (high correlation suggests they carry similar information) or if some variables are inversely related (negative correlation). It might also show clusters of variables – perhaps 3 variables all inter-correlated (forming a block) which could indicate they measure a similar underlying dimension of the business. Detecting multicollinearity (high correlation between independent variables) is an important part of EDA before building regression models, and a correlation matrix is the go-to tool for that.

Advanced Visualization: For very high-dimensional data, specialized EDA visualization methods exist. Parallel coordinates plots, for example, draw each observation as a line crossing a set of parallel axes (one axis per variable), allowing one to see patterns across many variables (though these can get cluttered for large datasets). Other methods include Chernoff faces (representing multivariate data as elements of a face – a more gimmicky approach), and various interactive tools where one can brush and link between plots. The key idea is to get any handle possible on how variables jointly behave. Often EDA might cycle between looking at raw variables (univariate), pairs, and then higher-order relationships, drilling down as new questions arise.

In practice, multivariate EDA is iterative and builds on the univariate and bivariate findings. For example, after seeing pairwise plots, an analyst might suspect an interaction (maybe variable X and Y relate differently depending on Z). They would then explicitly check that with a stratified plot (incorporating Z). Or if too many variables are involved to visualize directly, they might use PCA to reduce dimensions. Multivariate EDA is where the art of analysis often comes in – knowing which combinations to examine and which patterns to look for is gained by experience and domain knowledge.