About the RedditDayOf Archive

What?

RedditDayOf is a subreddit of the popular content aggregation site, Reddit.

Each day, interesting links about the day's topic are posted to the subreddit.

Why?

Unfortunately, due to the structure of Reddit, it is challenging to browse older topics directly.

I created this website to make it easier.

How?

Reddit has made available a REST API, which allows developers to access the content on the website.

Using PRAW, a python based wrapper for the Reddit API, I downloaded all of the posts for the RedditDayOf subreddit.

Unfortunately, the Reddit API limits all requests to the last 1000 posts. There is, however, a workaround that involves trawling through all subreddits - 100 posts at a time - and extracting only the posts for the subreddit in which you are interested.

I threw together this python script to mine for RedditDayOf posts, running it on a low-power, always-on computer. The Reddit API limits all applications to 1 request every 2 seconds and so, running 24 hours a day, it took three weeks to dig all the way to the birth of RedditDayOf in 2011.

Once mined, the data is imported into a SQL database to be "fixed" - unlabelled posts are assigned to the correct topic etc. - and uploaded to the website.

When?

Now that I have mined all the way to the birth of RedditDayOf, I have my script updating the website once a day with new posts. The intention of the RedditDayOf archive is to make it easier to browse topics that are older than one day, not to replace Reddit as the interface to RedditDayOf.

Who?

Everybody.

Back to RedditDayOf Archive