Learning Without Labels

Unsupervised learning is used when we have features ( $X$ ) but no target ( $y$ ). The goal is to find inherent patterns or structures within the data.

1. Clustering: Grouping Similar Samples

Clustering algorithms attempt to partition the data into groups (clusters) where samples in the same group are more similar to each other than to those in other groups.

K-Means Clustering

The most popular clustering algorithm. it partitions data into $K$ clusters by minimizing the distance between points and their cluster centroids.

Interactive Lab


from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs
import matplotlib.pyplot as plt

# Create 3 distinct blobs of data
X, _ = make_blobs(n_samples=300, centers=3, cluster_std=0.60, random_state=0)

# Instantiate and fit
kmeans = KMeans(n_clusters=3)
kmeans.fit(X)

# Predicted labels
y_kmeans = kmeans.predict(X)
print(f"Centroids:\n{kmeans.cluster_centers_}")

Expected output

Centroids:
[[ 0.94973332  4.41906906]
 [ 2.01527128  0.92212359]
 [-1.58438467  2.85831692]]

python

2from sklearn.cluster import KMeans

3from sklearn.datasets import make_blobs

4import matplotlib.pyplot as plt

6# Create 3 distinct blobs of data

7X, _ = make_blobs(n_samples=300, centers=3, cluster_std=0.60, random_state=0)

9# Instantiate and fit

10kmeans = KMeans(n_clusters=3)

11kmeans.fit(X)

13# Predicted labels

14y_kmeans = kmeans.predict(X)

15print(f"Centroids:\n{kmeans.cluster_centers_}")

2. Dimensionality Reduction

High-dimensional data (hundreds or thousands of features) is difficult to visualize and can lead to the “curse of dimensionality.” Dimensionality reduction seeks to represent data in a lower-dimensional space while preserving as much information as possible.

Principal Component Analysis (PCA)

PCA finds the “principal components”—the orthogonal axes along which the data varies the most.

Interactive Lab


from sklearn.decomposition import PCA
from sklearn.datasets import load_iris

iris = load_iris()
X = iris.data # 4 features

# Reduce from 4D to 2D
pca = PCA(n_components=2)
X_reduced = pca.fit_transform(X)

print(f"Original shape: {X.shape}")
print(f"Reduced shape: {X_reduced.shape}")
print(f"Explained variance ratio: {pca.explained_variance_ratio_}")

Expected output

Original shape: (150, 4)
Reduced shape: (150, 2)
Explained variance ratio: [0.92461872 0.05306648]

python

2from sklearn.decomposition import PCA

3from sklearn.datasets import load_iris

5iris = load_iris()

6X = iris.data # 4 features

8# Reduce from 4D to 2D

9pca = PCA(n_components=2)

10X_reduced = pca.fit_transform(X)

12print(f"Original shape: {X.shape}")

13print(f"Reduced shape: {X_reduced.shape}")

14print(f"Explained variance ratio: {pca.explained_variance_ratio_}")

3. Anomaly Detection

Identifying outliers or unusual patterns that do not conform to expected behavior.

Isolation Forest

Isolation Forest works by isolating anomalies using trees. Anomalies are easier to isolate and thus have shorter path lengths in the trees.

Interactive Lab


from sklearn.ensemble import IsolationForest
import numpy as np

# Generate normal data
X_train = 0.3 * np.random.randn(100, 2)
X_train = np.r_[X_train + 2, X_train - 2]

# Generate some abnormal novel observations
X_outliers = np.random.uniform(low=-4, high=4, size=(20, 2))

clf = IsolationForest(contamination=0.1)
clf.fit(X_train)

y_pred_outliers = clf.predict(X_outliers)
print(f"Predictions for outliers (-1 is anomaly):\n{y_pred_outliers}")

Expected output

Predictions for outliers (-1 is anomaly):
[-1 -1 -1  1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1  1]

python

2from sklearn.ensemble import IsolationForest

3import numpy as np

5# Generate normal data

6X_train = 0.3 * np.random.randn(100, 2)

7X_train = np.r_[X_train + 2, X_train - 2]

9# Generate some abnormal novel observations

10X_outliers = np.random.uniform(low=-4, high=4, size=(20, 2))

12clf = IsolationForest(contamination=0.1)

13clf.fit(X_train)

15y_pred_outliers = clf.predict(X_outliers)

16print(f"Predictions for outliers (-1 is anomaly):\n{y_pred_outliers}")

Summary of Unsupervised Techniques

Code

skinparam componentStyle rectangle

package "Unsupervised Tasks" {
component "Clustering" as CLUST
component "Dimensionality Reduction" as DIM
component "Density Estimation" as DENS
}

node "Algorithms" {
component "K-Means / DBSCAN / Agglomerative" as CALG
component "PCA / t-SNE / UMAP" as DALG
component "GMM / Kernel Density" as DEALG
}

CLUST --> CALG
DIM --> DALG
DENS --> DEALG

In the next section, we will explore specialized scientific libraries like SciPy and Matplotlib to round out our scientific Python toolkit.

Scikit-Learn: Unsupervised Learning

Learning Without Labels

1. Clustering: Grouping Similar Samples

K-Means Clustering

2. Dimensionality Reduction

Principal Component Analysis (PCA)

3. Anomaly Detection

Isolation Forest

Summary of Unsupervised Techniques