Building a fully automated Medium stats pipeline to track my writing performance

Combining Prefect, Selenium, Pandas and Metabase to measure and manage my writing success on Medium

Motivation and Background

1) Modern data stacks

2) From zero to hero — A marketing sophistication framework

You can’t manage what you don’t measure. — Peter Ducker

Building the Architecture

Target Architecture for a fully automated Medium stats pipeline

1. Scraping Medium Stats with Selenium

Showcasing of Medium Stats Scraper

2. Setting up the Extract, Transform Load Pipeline with Pandas

Medium Star Schema Model

3. Orchestrating workflows with Prefect

Example run of my medium stats scraper workflow.
Flow Run Schematics of Scraping Job.

4. Visualization of Results

Main Dahshboard VIew

1. Total Stats

Medium Main KPIs

2. KPI daily Series Changes

Times Series Stats on my Blogpost Performance
Before clicking on the Title.
After clicking on the Title.

Outlook and some even more exciting ideas

Total Medium Blogpost views by External viewing Source

A smart Twitter Bot

Operational Intelligence Business Analyst at N26 looking to build his skills in Data Science.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store