Machine learning is transforming industries, but at its core, it remains a field built on mathematics, logic, and structured data modeling. This article walks through the foundational principles of machine learning, stripping away the modern layers of abstraction to focus on its original essence. No libraries, no black boxes, just math and reasoning. Whether you’re a beginner with a math background or a curious technologist aiming to understand what actually powers intelligent systems, this is a ground-up journey into how machines learn.

Let’s rewind a bit. Before there were ready-to-use libraries like scikit-learn or TensorFlow, people still built intelligent systems. How? Through a solid understanding of math and statistics.

When I first approached machine learning, I took Andrew Ng’s course on Coursera. That course was instrumental in demystifying the basics. It is now hosted at DeepLearning.AI and remains one of the best starting points for anyone serious about understanding machine learning from first principles.

This post is inspired by that journey. It’s for readers who want to understand how models learn from data and improve their performance over time, without relying on external libraries or tools. Just clean, raw ideas.

What is Machine Learning?

At its core, machine learning is the study of algorithms that improve automatically through experience. That definition can feel a bit abstract, so let’s anchor it with a more rigorous interpretation.

A computer program is said to learn from experience (E) with respect to some task (T) and performance measure (P), if its performance at T, as measured by P, improves with E.

In other words, we want a machine to:

• Perform a task (T)
• Learn from data or experience (E)
• Get better over time based on a measurable metric (P)

For example, suppose we want a model to predict housing prices (T), and we give it historical housing data (E). If its predictions get more accurate over time, as measured by mean squared error (P), then it is learning.

The Mathematical Foundations

Machine learning is grounded in four core areas of mathematics. Let’s explore each of these and how they relate to the learning process.

Linear Algebra

Linear algebra is the foundation of data representation in machine learning.

• Vectors represent individual data points. For example, a single house might be represented by a vector containing its size, number of rooms, and age.
• Matrices represent entire datasets. Each row is a data point, and each column is a feature.
• Dot products and matrix multiplications are used to compute predictions and update models.

Example

In linear regression, the predicted value ŷ is calculated as:

ŷ = Xβ

Here:

• X is the feature matrix
• β is the vector of model parameters
• ŷ is the vector of predicted outputs

This operation is fundamentally a matrix multiplication, and its simplicity hides its power.

Calculus

Calculus is used to optimize models, meaning we adjust the parameters to minimize the error.

• Derivatives help us understand how small changes in parameters affect the model’s predictions.
• Gradients are vectors of partial derivatives. They show the direction and rate of fastest increase in a function.
• Gradient descent uses these gradients to iteratively reduce error by updating parameters in the opposite direction of the gradient.

Formula

Gradient Descent Update Rule:

θ = θ - α * ∇J(θ)

Where:

• θ are the model parameters
• α is the learning rate
• ∇J(θ) is the gradient of the cost function

This process is repeated until the cost function reaches a minimum.

Probability and Statistics

Machine learning deals with uncertainty. Probability helps us model and reason about that uncertainty.

• Random variables model unknown outcomes
• Distributions describe how values are spread
• Bayesian inference lets us update beliefs in light of new evidence

Example

In logistic regression, we model the probability that an output belongs to a certain class:

P(y = 1 | x) = 1 / (1 + e^(-wᵀx))

This probability is derived from the logistic function and gives us a smooth way to transition between class 0 and class 1.

Optimization

Once you define how to measure error, optimization is how you reduce it. Most learning algorithms boil down to an optimization problem.

Common methods include:

• Gradient descent
• Stochastic gradient descent
• Convex optimization, which ensures global optima under specific conditions

Each machine learning algorithm defines an objective (or cost) function and uses optimization to find parameters that minimize it.

Categories of Machine Learning

Machine learning problems are typically grouped into three major categories: supervised learning, unsupervised learning, and reinforcement learning. Each category is defined by the nature of the data and the goal of the learning process.

Supervised Learning

Supervised learning is the most widely used category. It involves learning a mapping from inputs to known outputs using labeled data.

Formal Setup

Given a dataset of input-output pairs:

D = {(x₁, y₁), (x₂, y₂), ..., (xₙ, yₙ)}

The goal is to learn a function f(x) such that f(xᵢ) ≈ yᵢ for all i.

Subtypes

• Regression: Predicting continuous values (e.g., house prices)
• Classification: Predicting discrete labels (e.g., spam or not spam)

Mathematical Example: Linear Regression

The model assumes:

ŷ = wᵀx + b

Where:

• w is the weight vector
• b is the bias term
• x is the feature vector

The cost function (Mean Squared Error):

J(w, b) = (1/n) * Σ(yᵢ - ŷᵢ)²

To minimize this, we compute gradients and apply gradient descent.

Key Insight

The learning process involves adjusting weights so that the predictions get closer to the actual labels. The algorithm “learns” by reducing error.

Unsupervised Learning

In unsupervised learning, the data is not labeled. The goal is to find hidden patterns, groupings, or structures.

Formal Setup

Given data points:

D = {x₁, x₂, ..., xₙ}

The task is to model the underlying structure or distribution.

Subtypes

• Clustering: Grouping similar items together
• Dimensionality Reduction: Reducing the number of variables while preserving structure

Key Insight

Unsupervised learning models do not have correct answers. Instead, they discover structure based on similarity, density, or statistical properties.

Reinforcement Learning

Reinforcement learning is about learning to make decisions. An agent interacts with an environment and learns from the consequences of its actions.

Formal Setup

The agent receives:

• State s
• Action a
• Reward r
• New state s'

Its goal is to learn a policy π(a|s) that maximizes the expected cumulative reward:

E[Σ γᵗ * rₜ]

Where:

• γ is the discount factor
• rₜ is the reward at time t

Key Insight

Reinforcement learning is like trial and error guided by rewards. The agent learns what to do by interacting with the environment and observing outcomes.

Real-World Examples

• Supervised learning is like learning with an answer key. You see the correct answers and adjust based on feedback.
• Unsupervised learning is like exploring a new city without a map. You group neighborhoods by feel and pattern.
• Reinforcement learning is like training a dog. Correct actions are rewarded, and the dog learns what earns treats over time.

Building a Solid Machine Learning Mindset

You don’t need a PhD to understand machine learning. But what you do need is a mindset that values structure, clarity, and fundamentals. Getting comfortable with the mathematical building blocks is the first step toward real mastery and it pays dividends whether you’re debugging a model, reading a research paper, or just evaluating claims in a product pitch.

Why the Basics Matter

It’s tempting to jump straight into tools and frameworks. But tools change; math does not. A linear regression model today looks just like it did 30 years ago. The same goes for gradient descent, cost functions, and probability distributions. By grounding yourself in the core principles:

• You gain intuition for how models behave
• You can debug more effectively when things go wrong
• You avoid being misled by flashy but shallow explanations

How to Keep Learning

If this post sparked your interest, here are a few great next steps that stay grounded in fundamentals:

• Revisit high-school and college-level math topics like algebra, calculus, and statistics
• Go through Andrew Ng’s Machine Learning Specialization at DeepLearning.AI, which provides clear explanations without jumping too fast into coding
• Practice by writing out algorithms on paper, simulate how gradient descent works step by step
• Follow mathematically inclined blogs like StatQuest with Josh Starmer or Machine Learning Mastery for practical breakdowns

Summarizing

Machine learning is not a magic spell, it’s a collection of mathematical techniques applied to data to make predictions, discover patterns, or guide decisions. We explored the roots of ML in linear algebra, calculus, probability, and optimization. We examined the three main categories: supervised, unsupervised, and reinforcement learning, and grounded each one in its mathematical framework.

This back-to-basics perspective reveals something powerful: you don’t need massive datasets or cutting-edge hardware to start understanding machine learning. You just need curiosity, a bit of math, and the willingness to think deeply about how systems learn from data.

References

• Andrew Ng’s Machine Learning Specialization
• StatQuest
• Machine Learning Mastery
• MIT OpenCourseWare: Introduction to Deep Learning