Streaming your database - Easier said than done?
Do you need to stream your database? So did I! Here’s what to expect.
In my journey as a contractor for a major cybersecurity company, I implemented a Change Data Capture (CDC) system that aimed to seamlessly stream client databases into a centralized cloud solution for analytics and enhanced features. However, after deploying the code, I quickly realized that the reality was far more complex than anticipated.
Despite using an off-the-shelf product, I encountered unexpected challenges related to scaling, data formats, integrations, and deployment—issues that took me by surprise and required significant adjustments.
Over the course of 18 months, my team and I navigated these hurdles to create a robust system designed for scalability and resilience against failures.
In this talk, I’ll share key lessons learned and practical tips for developers and architects embarking on similar projects. You’ll gain insights into preparing for potential surprises and overcoming challenges in building a reliable CDC system.
Sigal Shaharabani
