How to build Algorithm for Social Media Feed — The Basics & System design (Part 1)
Around a week ago I was assigned the task “we need a feed like Twitter or linkedin”, I had zero idea how this would be done.
Feeds look simple from the outside… until you try to make one that’s fast, personalized, doesn’t cost a fortune, and doesn’t die when someone goes viral.
This is Part 1 of how we actually built it .
Feeds are secretly one of the hardest parts of any social app
Everyone wants the same dream:
- Loads in under 1 second (ideally <100 ms)
- Shows posts from people you actually follow
- Throws in some fresh viral content
- Doesn’t show the same 3 people 20 times in a row
- Doesn’t cost $10k/month when you reach 100k users
Turns out… you can’t get all of that with one naive solution. There are painful trade-offs everywhere.
The two classic ways people build feeds (and why both hurt alone)
1. Push everything when someone posts (“fan-out on write”)
When Maria posts a cute dog video → we immediately copy that post to the personal “timeline” of every single follower she has.
Pros
Reading the feed later is lightning fast — just read your own list!
Cons
If Maria suddenly gets 500,000 followers? You have to write the same post 500,000 times.
Servers scream. Database bills explode. 😭
2. Pull everything when the user opens the app (“pull on read”)
When you refresh the feed → the system goes:
“Okay… find the latest posts from the 312 people Anna follows + some popular stuff right now.”
Pros
Super safe for celebrities. One viral post = one write. No problem.
Cons
If you follow 500+ active people, every refresh becomes a heavy search. Latency kills the experience.
What we actually do: a hybrid push + pull system
Normal people → push
Celebrities → pull
Simple rule that changed everything.
Why treat some users as “celebrity” ?
Here’s the thing that made my brain click:
We treat users completely differently depending on follower count.
< ~1000 followers → we push (copy) the post to all followers right now
≥ 1000 followers → we DO NOT copy. We just save it in the author’s own “outbox”
We call anyone ≥1000 followers a “celebrity” (yes, even if it’s just your friend who posts gym mirror selfies every day 😂)
Why 1000?
It’s the magic number we found after testing.
Small enough → pushing is still fast.
Big enough → we don’t murder the database the moment someone goes viral.
Where everything actually lives (mostly Redis)
We lean very heavily on Redis because it’s stupidly fast for this kind of workload.
Here are the most important keys we use every day:
- timeline:anna123 → Anna’s normal feed (only posts from regular-sized accounts)
- celebrity_posts:bob789 → Bob’s own posts (only exists if Bob is “celebrity”)
- post:uuid-abc123-def456 → cached full content of one post (lives ~1 hour)
- celebrities → simple set containing all user IDs with ≥1000 followers
- trending_posts → the current hot/viral posts everyone might see
What really happens when someone posts?
Case 1: Normal person posts (<1000 followers)
Maria posts cute cat → we get her followers (usually <300)
Then we do: ZADD timeline:john ... post-id
ZADD timeline:sarah ... post-id
ZADD timeline:mike ... post-id
...
Like sending the same postcard to 300 friends. Fast. Cheap. Done.
Case 2: Celebrity posts (≥1000 followers)
MrBeast posts → we ONLY do:
ZADD celebrity_posts:mrbeast timestamp post-id
One single write. Extremely cheap.
We save the heavy work for later — when people actually want to read his stuff.
Quick recap — what we covered in Part 1
- Normal accounts → push posts to followers → reading becomes fast
- Big accounts (“celebrities”) → pull on read → doesn’t explode on viral posts
- Almost everything important lives in Redis sorted sets (perfect for “latest first” timelines)
- We automatically trim old posts (keep ~500 latest per user)
Some References
- Twitter’s Timeline Architecture-https://blog.twitter.com/engineering/en_us/topics/infrastructure/2017/the-infrastructure-behind-twitter-scale
- https://www.infoq.com/presentations/Twitter-Timeline-Scalability/
- Designing a News Feed (System Design)-https://www.educative.io/courses/grokking-the-system-design-interview
In Part 2 we’ll get to the really fun part: How we actually decide what shows up first, how we mix friends + viral posts, how we avoid boring repetitive feeds, and how guest users get a feed without even logging in.
If you enjoyed this and excited for Part 2, feel free to follow me here on Medium — or come say hi on X/Twitter @manujdxt 😄 I usually post small dev tips, architecture breakdowns, and the occasional confused dev moment.
Thanks for reading — see you in Part 2 next week! 🚀