Home/Learn/Numerical Methods & Optimization

Pattern Guide

Numerical Methods & Optimization

"Binary search on reals. Ternary search on unimodal functions. Simulated annealing for hard optimizations."

Numerical methods for competitive programming: binary search on real-valued answers, ternary search on unimodal functions, Newton's method, and simulated annealing for continuous optimization problems.

Problems you can solve with this pattern

5 problems · click any to start solving

All num methods
1Kth Smallest Number in Multiplication TableHardSolve
2Find the Smallest Divisor Given a ThresholdMediumSolve
3Minimize Max Distance to Gas StationHardSolve
4Magnetic Force Between Two BallsMediumSolve
Simulated annealing template
function simulatedAnnealing(initialState, energy, neighbor, maxIter = 100000) {
    let state = initialState;
    let bestState = state;
    let bestEnergy = energy(state);
    let T = 1.0; // initial temperature
    const cooling = 0.9999;

    for (let i = 0; i < maxIter; i++) {
        T *= cooling;
        const next = neighbor(state);
        const delta = energy(next) - energy(state);
        // Accept if better, or with probability e^(-delta/T) if worse
        if (delta < 0 || Math.random() < Math.exp(-delta / T)) {
            state = next;
            if (energy(state) < bestEnergy) {
                bestState = state;
                bestEnergy = energy(state);
            }
        }
    }
    return bestState;
}

// Example: minimize sum of distances from point to set of points (geometric median)
// neighbor: small random perturbation of (x, y)
// energy: sum of euclidean distances

Numerical methods bridge continuous math and discrete algorithms. The core skill: recognizing when an answer is real-valued (not integer) and which search/optimization technique applies. Binary search on reals converges in ~50 iterations to any required precision. Ternary search finds the minimum/maximum of a unimodal function. Simulated annealing escapes local optima for NP-hard problems.

Binary Search on Reals

Pattern: "Find smallest/largest real x such that f(x) is true/false." f must be monotone.

Convergence: 100 iterations gives precision ~10⁻³⁰. Usually 50 iterations is enough for any problem.

Common uses: find radius such that k circles cover all points, find time t such that tasks complete, find speed such that journey is possible.
Binary search on real-valued answer
// Template: find smallest x in [lo, hi] where check(x) is true
function binarySearchReal(lo, hi, check, iterations = 100) {
    for (let i = 0; i < iterations; i++) {
        const mid = (lo + hi) / 2;
        if (check(mid)) hi = mid;
        else lo = mid;
    }
    return (lo + hi) / 2;
}

// Example: minimum radius so k circles cover all n points on a line
function minRadius(points, k) {
    points.sort((a, b) => a - b);
    const check = (r) => {
        let covered = 0, circles = 0;
        while (covered < points.length) {
            circles++;
            const start = points[covered];
            // circle covers [start, start + 2r]
            while (covered < points.length && points[covered] <= start + 2*r)
                covered++;
        }
        return circles <= k;
    };
    return binarySearchReal(0, points[points.length-1]-points[0], check);
}

Ternary Search (Unimodal Functions)

Unimodal: strictly decreasing then increasing (or vice versa) — one minimum/maximum.

Ternary search: at each step, evaluate f at m1=(lo+2lo+hi)/3 and m2=(lo+hi+2hi)/3. Eliminate the third where the function value is worse.

Convergence: each iteration shrinks range by 1/3. Need ~200 iterations for float precision.
Ternary search — find minimum of unimodal function
// Find x in [lo, hi] minimizing f(x), where f is unimodal (convex)
function ternarySearch(lo, hi, f, iterations = 200) {
    for (let i = 0; i < iterations; i++) {
        const m1 = lo + (hi - lo) / 3;
        const m2 = hi - (hi - lo) / 3;
        if (f(m1) < f(m2)) hi = m2;
        else lo = m1;
    }
    return (lo + hi) / 2;
}

// Example: find point on line segment closest to origin
// f(t) = distance from point (t*ax + (1-t)*bx, t*ay + (1-t)*by) to (0,0)
function closestPointOnSegment(ax, ay, bx, by) {
    const f = t => {
        const x = ax + t*(bx-ax), y = ay + t*(by-ay);
        return x*x + y*y;
    };
    const t = ternarySearch(0, 1, f);
    return [ax + t*(bx-ax), ay + t*(by-ay)];
}

// Integer ternary search
function ternarySearchInt(lo, hi, f) {
    while (hi - lo > 2) {
        const m1 = Math.floor((lo*2 + hi) / 3);
        const m2 = Math.floor((lo + hi*2) / 3);
        if (f(m1) <= f(m2)) hi = m2;
        else lo = m1;
    }
    let best = lo;
    for (let x = lo+1; x <= hi; x++) if (f(x) < f(best)) best = x;
    return best;
}

Simulated Annealing

When to use: problem is NP-hard or has no clean greedy/DP solution. Accepts slightly worse solutions with probability e^(-delta/T) to escape local optima. Temperature T decreases over time (cooling schedule).

Common in CP: TSP variants, continuous coordinate optimization, problems where brute force times out and heuristic is acceptable.
Simulated annealing template
function simulatedAnnealing(initialState, energy, neighbor, maxIter = 100000) {
    let state = initialState;
    let bestState = state;
    let bestEnergy = energy(state);
    let T = 1.0; // initial temperature
    const cooling = 0.9999;

    for (let i = 0; i < maxIter; i++) {
        T *= cooling;
        const next = neighbor(state);
        const delta = energy(next) - energy(state);
        // Accept if better, or with probability e^(-delta/T) if worse
        if (delta < 0 || Math.random() < Math.exp(-delta / T)) {
            state = next;
            if (energy(state) < bestEnergy) {
                bestState = state;
                bestEnergy = energy(state);
            }
        }
    }
    return bestState;
}

// Example: minimize sum of distances from point to set of points (geometric median)
// neighbor: small random perturbation of (x, y)
// energy: sum of euclidean distances

Newton's Method

Newton's method — find root of f(x) = 0
// Fast square root via Newton's method
function sqrtNewton(n) {
    if (n === 0) return 0;
    let x = n;
    while (true) {
        const next = (x + n/x) / 2;
        if (Math.abs(next - x) < 1e-9) return next;
        x = next;
    }
}

// General Newton iteration: x_{n+1} = x_n - f(x_n)/f'(x_n)
// Converges quadratically (doubles correct digits each step)
function newtonMethod(f, fPrime, x0, tolerance = 1e-9) {
    let x = x0;
    for (let i = 0; i < 100; i++) {
        const dx = f(x) / fPrime(x);
        x -= dx;
        if (Math.abs(dx) < tolerance) break;
    }
    return x;
}
Choosing the right method:
- Answer is an integer, condition is monotone → binary search (exact)
- Answer is a real number, condition is monotone → binary search (100 iters)
- Find min/max of unimodal continuous function → ternary search (200 iters)
- Need fast root finding → Newton's method (~10 iters)
- "Maximize minimum" or "Minimize maximum" → binary search on answer + greedy check

Precision rule: 100 iterations of binary search on [0, 1e9] gives ~10⁻²¹ precision. Way more than enough.