How linear regression works intuitively and how it leads to gradient descent

Summary

Learning, to a computer, is just turning bad guesses into better ones. In this post, we’ll see how that starts with a straight line: how linear regression makes the first guess, and gradient descent keeps improving it. Let's start with something familiar: house prices. Bigger houses tend to cost more; smaller ones, less. It's the kind of pattern you can almost see without thinking: more space, more money. When we plot it, the shape is clear: a loose upward slope, with some noise but a definite trend. As you can see, price and size move together in a way that feels predictable. Not in fixed steps or categories, but on a sliding scale. A house might go for $180,000, $305,500, or anything in between. Now imagine you're selling your own house. It's 1,850 square feet—larger than average, but not a mansion. You've seen what homes go for in your area, but the prices are scattered. What's a fair number to list it at? One option is to text your real estate friend and get a half-baked guess. A better option is to look at the pattern in past sales and sketch a line that seems to match it. You grab a ruler, hold it up to the scatterplot, and draw something that feels about right. Then you find your square footage on the x-axis, trace upward to your line, and read off the predicted price. Whatever line you draw, it'll be a steady upward slope. Bigger homes, higher prices. It might not be perfect, but it gives you a way to turn square footage into a price that kind of makes sense. And while the lines you might draw vary, they all follow the same formula. Each one takes the area of a house (the explanatory variable), multiplies it by a number (the slope), and adds another number (the intercept). Notice that this isn't just a house-pricing formula. It's how every straight line works. One number sets the tilt (the slope), and the other shifts the line up or down (the intercept). That's all it takes to draw a line: scale, then slide. In our case, the slope is the "price-per-square-fo...