Product
Blog /

What is Decodable?

Riley Johnson
Decodable

In one conversation over another, the Decodable team has found ourselves explaining “What is Decodable?”  Is it like Kafka?  Is it like Fivetran?  Is it like kSQL?

In short, it's the easiest way to build the best pipelines you’ve ever built. But the long answer is there isn’t a category that Decodable fits nicely into because it’s so unlike previous iterations of data integration tools.

Over a two month period, the Decodable team interviewed 30+ tech executives from firms large and small, asking “If you had the perfect product for moving and transforming data, what would it look like?”  The same set of requirements surfaced again and again.

First, it would be in roughly the same category as Flink or Spark Streaming, but as-a-service. That toolset category has its own requirements, like dynamic scaling, fault tolerance, state store, recovery, etc.  By saying this, the customers were expressing that they want a compute-as-a-service engine, not a java library.  

Here are the requirements we heard:

  • It will definitely be a cloud service, and it has to be a serverless experience - like AWS Lambda.
  • Because their firms each have a mix of AWS, GCP, Kafka, Kinesis, Confluent, Pulsar, Redpanda, etc, it has to be interoperable between vendors, technologies, and clouds.
  • Ease of use is paramount.  It can support java or scala, but it must support SQL.
  • It must be multitenant and secure by design.
  • It has to be manageable at scale. In other words, it has to provide seamless data governance capabilities: data lineage, data catalog, schema management.
  • It can’t become a silo of dark code, it has to be integrated with natural-feeling code workflows.
  • It has to have a full Tier 1 Support model with an SLA.  It can’t be treated the same “Tier 3 Support” way analytics ETL tools are.

Unlike Lambda which is essentially a blank slate for your functions, Decodable is more guided and opinionated.  Decodable aspires to interpret your intentions and get you 90% of the way to having a pipeline built from word go.

Taken together, all of this probably puts Decodable in the same category as things like Kinesis Data Analytics, but KDA requires a bunch of tools (Kinesis, KDA Flink, Glue, Zeppelin) to cover roughly the same surface area, and it isn’t interoperable and it isn’t easy to use. It might include kSQL if you squint, but kSQL isn’t serverless and isn’t interoperable.

This space is new and exciting and we anticipate new market entrants every day.  It will be a lot of fun to watch this category develop as innovations add more requirements to this list!

The Decodable team formed around a common set of problems we saw at customers, and are on a journey to find the easiest way to explain what we've built when it doesn't easily fit any pre-existing categories.

How would you answer the question "What is Decodable?" Drop into the community Slack or message us on Twitter or LinkedIn with your ideas - there may be some swag for the best responses!


You can get started with Decodable for free - our developer account includes enough for you to build a useful pipeline and - unlike a trial - it never expires.

Learn more:

Join the community Slack

Keeping Things Simple

Why hasn’t stream processing experienced hockey stick growth that streaming has? In this post we explore what's behind this lag and why it's about to rapidly change.

Learn more

We’re Abusing The Data Warehouse; RETL, ELT, And Other Weird Stuff.

By now, everyone has seen the rETL (Reverse ETL) trend: you want to use data from app #1 to enrich data in app #2. In this blog, Decodable's founder discusses the (fatal) shortcomings of this approach and how to get the job done.

Learn more

Flink Deployments At Decodable

Decodable’s platform uses Apache Flink to run our customers’ real-time data processing jobs. This blog post explores how we securely, reliably and efficiently manage the underlying Flink deployments at Decodable in a multi-tenant environment.

Learn more

Heading

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.

Learn more
Tags
Pintrest icon in black
Ideas & Opinions

Start using Decodable today.