Yakread's Ranking Algorithm

Summary

Continuing the Yakread rewrite, last weekend I rewrote the part of Yakread's ranking algorithm that merges your newsletter/RSS subscriptions and your bookmarked articles into a single personalized feed. This is basically the “heart” of Yakread, the whole reason I wanted to make the app in the first place. (And it's about 200 lines of code. Funny how much additional stuff has to be layered on top just to make those 200 lines of code usable.) This part of the algorithm does three main things: it picks which subscription posts to recommend; it picks which bookmarked articles to recommend; and it decides in what order to mix those posts and articles together. The end result is a batch of, say, 30 items which will be displayed to the user on their For You feed. (Instead of “posts” and “articles”, I typically use the more general term “item”, which is a common term for things that recommender systems recommend.) Bookmarks First we take all your unread bookmarked items and sort them by how many times you've scrolled past them previously. For items with the same number of “skips,” we sort by how recently the item was bookmarked. “Fresher” items (those that have been skipped less and were bookmarked more recently) come first. To shake things up a bit, we then partially shuffle the items. I wrote a simple shuffle algorithm that has a bias toward leaving items close to their original spots. There's a “randomness” parameter (I'll call it p) you can set between 0 and 1: 0 will make the algorithm behave like a completely random shuffle, and 1 will leave the items in their original order. I've set p to 0.1. That might make it sound like the items will be mostly-but-not-completely in random order, minimizing the impact of the initial sort-by-freshness. However, the parameter affects the randomness in a non-linear way. The following graph approximately shows the probability that the first item in the shuffled list will be one of the first ten items in the original list (y axis), giv...