Your browser doesn't support the features required by impress.js, so you are presented with a simplified version of this presentation.

For the best experience please use the latest Chrome, Safari or Firefox browser.

The Krihelinator
Evaluating trendiness in open source software

Tom Gurion

Github's trending page github trending


github trending

Trendiness of OSS should be assessed by contribution rate, not by stars

Meir Kriheli
Enters the Github Pulse page PokemonGo-Map pulse page
We can't sort projects by contribution rate... the krihelinator


20  * authors
8  * merged and proposed pull requests
8  * new and closed issues
1  * commits

Alternatives openhub

Anything else?

Pipeline process

pipeline diagram


Poll the Github API to get all of the repositories on Github, in a loop, 100 at a time.

16 Scrapers

Scrape the Pulse page for each repo, extract last week contribution statistics.



Calculate Krihelimeter and persist to DB.

Periodic process

Kicks in every 6 hours.
Scrapes the Github trending page, pass to the scrapers.
Keeps the 500 repositories with the highest Krihelimeter, deletes the rest.
Rescrapes and updates all of the repositories in the DB.
elixir logo
Simpler concurrency model
Fault tolerance

Higher level language

Less infrastructure

No job queue, no workers, no celery...

More throughput