The Scikit-Learn Ecosystem

Scikit-Learn (frequently abbreviated as sklearn) is the primary library for classical machine learning in Python. Built on top of NumPy, SciPy, and Matplotlib, it provides simple and efficient tools for predictive data analysis.

Key Concepts and the Estimator API

The brilliance of Scikit-Learn lies in its consistent API. Whether you are performing linear regression, support vector machines, or random forests, the workflow is almost identical.

skinparam componentStyle rectangle

package "Data Preparation" {
[Features (X)]
[Target (y)]
}

node "Scikit-Learn Workflow" {
component "Instantiate Model" as MK
component "fit(X, y)" as FIT
component "predict(X_new)" as PRED
}

[Features (X)] --> MK
[Target (y)] --> MK
MK --> FIT : "Training"
FIT --> PRED : "Inference"

The Three Main Steps:

1. Instantiate: Choose your model class and set hyperparameters.
2. Fit: Train the model on your data using the fit() method.
3. Predict: Apply the trained model to new data using predict() or transform().

Data Representation in Scikit-Learn

Scikit-Learn expects data in a specific format:

• X (Feature Matrix): A 2D array or DataFrame of shape [n_samples, n_features].
• y (Target Vector): A 1D array or Series of length n_samples.

A Simple Example: Linear Regression

Let’s see how we can predict a continuous value using Scikit-Learn.

from sklearn.linear_model import LinearRegression
import numpy as np

# 1. Create Data
X = np.array([[1], [2], [3], [4]]) # Feature matrix
y = np.array([2, 4, 6, 8])         # Target vector (y = 2*x)

# 2. Instantiate Model
model = LinearRegression()

# 3. Fit Model
model.fit(X, y)

# 4. Predict
X_new = np.array([[5], [10]])
predictions = model.predict(X_new)

print(f"Predictions for 5 and 10: {predictions}")
print(f"Coefficient (Slope): {model.coef_[0]}")
print(f"Intercept: {model.intercept_}")

Predictions for 5 and 10: [10. 20.]
Coefficient (Slope): 2.0
Intercept: 0.0

The Machine Learning Workflow

A real-world project involves more than just fitting a model. It requires rigorous evaluation.

Train/Test Splitting

We must never evaluate a model on the same data it was trained on. Scikit-Learn provides train_test_split to handle this.

from sklearn.model_selection import train_test_split
from sklearn.datasets import load_iris

# Load sample data
iris = load_iris()
X, y = iris.data, iris.target

# Split into 80% training, 20% testing
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

print(f"Training set size: {X_train.shape[0]}")
print(f"Testing set size: {X_test.shape[0]}")

Training set size: 120
Testing set size: 30

Preprocessing: Scaling and Encoding

Machine learning models are sensitive to the scale of features. For example, a model might give more weight to a feature ranging from 0 to 1000 than to one ranging from 0 to 1.

from sklearn.preprocessing import StandardScaler
import numpy as np

data = np.array([[10, 0.001], [20, 0.002], [30, 0.003]])
scaler = StandardScaler()

scaled_data = scaler.fit_transform(data)
print("Original Data:\n", data)
print("\nScaled Data (mean=0, std=1):\n", scaled_data)

Original Data:
 [[1.e+01 1.e-03]
 [2.e+01 2.e-03]
 [3.e+01 3.e-03]]

Scaled Data (mean=0, std=1):
 [[-1.22474487 -1.22474487]
 [ 0.          0.        ]
 [ 1.22474487  1.22474487]]

Evaluation Metrics

How do we know if our model is any good? Scikit-Learn offers a suite of metrics.

• For Regression: Mean Squared Error (MSE), R-squared.
• For Classification: Accuracy, Precision, Recall, F1-Score.

from sklearn.metrics import accuracy_score

# Simulated ground truth and predictions
y_true = [0, 1, 2, 0, 1]
y_pred = [0, 2, 1, 0, 1]

acc = accuracy_score(y_true, y_pred)
print(f"Model Accuracy: {acc * 100:.1f}%")

Model Accuracy: 60.0%

In the next module, we will explore supervised learning models like Decision Trees and Random Forests in much greater detail.