I have always been interested in learning more about consumer applications like building a chat room, this weekend, I managed to build a chat room using Kafka and Python flask just in a few hours.
As you can tell from a consumer who has been subscribing to the topic, all the messages are now in the queue. The idea is whenever there is a new message, there is a mechanism to detect and then “broadcast” or “publish” to all the “subscribers”, or all corresponding clients who happened to be in the same chat room.
At this moment, I don’t know how to create a long running process within Flask so I created a small Python consumer running in another script. It listens to the topic and push any new message to another Flask endpoint called “broadcast”.
The broadcast endpoints does nothing more than taking in the new message, but it will publish it via server side event. There are plenty of benefits of using server side event comparing with polling or sometimes even a open websocket.
Now, all the clients have an event listener to the SSE stream. The any received new message from the stream will trigger the newly received message be appended to the chat history.
More work can be done around the Kafka part as the user’s chat history could be retrieved from a database or we use Kafka as the database directly. And the offset will determine to how long a history that we will go back.
Anyway, it is a pretty fun project overall. 🙂