.svg)
.svg)
Postgres CDC Overview
PostgreSQL is a powerful, open source object-relational database system that uses and extends the SQL language combined with many features that safely store and scale the most complicated data workloads. The origins of PostgreSQL date back to 1986 as part of the POSTGRES project at the University of California at Berkeley and has more than 30 years of active development on the core platform.
PostgreSQL has earned a strong reputation for its proven architecture, reliability, data integrity, robust feature set, extensibility, and the dedication of the open source community behind the software to consistently deliver performant and innovative solutions. PostgreSQL runs on all major operating systems, has been ACID-compliant since 2001, and has powerful add-ons such as the popular PostGIS geospatial database extender. It is no surprise that PostgreSQL has become the open source relational database of choice for many people and organizations.
The traditional approach to syncing PostgreSQL with complementary data stores is batch-based, where pipelines extract data from PostgreSQL and send it to downstream data stores. Change data capture (CDC) is a modern alternative to inefficient bulk imports. CDC extracts change events (INSERTs, UPDATEs, and DELETEs) in real-time from data stores, such as PostgreSQL, and provides them to downstream systems which can be applied to a copy of the data.
Decodable + Postgres CDC
Decodable can take input from a Postgres database as a source using the Postgres CDC (Change Data Capture) connector which converts table changes into a stream of change records. Decodable can be a super convenient means to replicate the contents of a Postgres database into a range of systems including messaging such as Kafka, analytical databases such as Apache Pinot, or machine learning feature stores. Decodable's SQL-on-the-stream transformation process also includes data virtualization on change streams so you're processing against the table, not the change records.
In this video, Gunnar Morling demonstrates how to set up a Postgres database on Amazon RDS for change data capture, so that you can emit data change streams to tools like Debezium or managed stream processing platforms such as Decodable.
In this video series, we show how to ingest data via Postgres CDC, process it on the Decodable platform, and load it back into Postgres and other external systems.
.svg)