Where is my bus?

Real-time Maps with Kafka, Python and Leaflet

Building a live map of London showing current bus line locations.

Image for post
Image for post
The world map (Source).

Everybody is talking about real-time data nowadays and indeed the need is real. Real-time availability of data is becoming a crucial success factor and driver for digitalization in nearly all areas like business analytics, financial data, cybersecurity, in the public sector, and so on.

Let’s have a closer look at two examples of which we will implement the second one in this blog series.

(1) Financial data, Real-time fraud detection

Your bank for sure owns algorithms that can protect your account in detecting fraud transactions. In the past, those algorithms were scheduled once a day giving a bulk output of possible fraud transactions. Employees need to take care of reversing malicious transactions with huge manual efforts and attackers may have caused already large damage.

The possibility to detect and stop fraud transactions in real-time as they appear would save huge manual efforts and limit the potential damage attackers could cause. Fortunately, real-time fraud detection is possible nowadays.

(2) Public sector, Real-time maps

Imagine you are on a sightseeing trip in— let’s take London — and you want to take a bus to visit an attraction like Buckingham Palace. Imagine the city offers an App that provides a live map showing the routes & locations of the current bus lines. You could use this map to locate the next best bus line and hop-on-stop to your desired destination.

This is the example we are creating during this series — a map of London with real-time location updates of three different bus lines as shown in the below picture and video.

Image for post
Image for post

We’ll make use of the following technologies:

  • Apache Kafka as high performant low latency event streaming platform
  • Python Pykafka Client to write Kafka Producers and Consumers
  • Python Flask as webserver and for API setup
  • Javascript Leaflet.JS for visualization of the real-time map
  • HTML5 Server-Sent Events to enable the web page to automatically get updates from the Webserver/Kafka Consumer

Additionally, if you are interested in analyzing Twitter data in real-time, you can check out my recent article.

I will split this story into a series of overall 4 stories as shown in the below overview. This allows me to get into more detail of each specific part of the overall solution. As this is my first medium post ever, I would highly appreciate any kind of feedback.

Have fun :-)

IT Architect with a passion for Marketing & Sales. Tech Enthusiast & Dog Lover. Buy me a coffee :-) https://www.buymeacoffee.com/codeanddogs

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store