Back
May 19, 2022
4
min read

What is Decodable?

By
Decodable Team
Share this post

In one conversation over another, the Decodable team has found ourselves explaining “What is Decodable?”  Is it like Kafka?  Is it like Fivetran?  Is it like kSQL?

In short, it's the easiest way to build the best pipelines you’ve ever built. But the long answer is there isn’t a category that Decodable fits nicely into because it’s so unlike previous iterations of data integration tools.

Over a two month period, the Decodable team interviewed 30+ tech executives from firms large and small, asking “If you had the perfect product for moving and transforming data, what would it look like?”  The same set of requirements surfaced again and again.

First, it would be in roughly the same category as Flink or Spark Streaming, but as-a-service. That toolset category has its own requirements, like dynamic scaling, fault tolerance, state store, recovery, etc.  By saying this, the customers were expressing that they want a compute-as-a-service engine, not a java library.  

Here are the requirements we heard:

  • It will definitely be a cloud service, and it has to be a serverless experience - like AWS Lambda.
  • Because their firms each have a mix of AWS, GCP, Kafka, Kinesis, Confluent, Pulsar, Redpanda, etc, it has to be interoperable between vendors, technologies, and clouds.
  • Ease of use is paramount.  It can support java or scala, but it must support SQL.
  • It must be multitenant and secure by design.
  • It has to be manageable at scale. In other words, it has to provide seamless data governance capabilities: data lineage, data catalog, schema management.
  • It can’t become a silo of dark code, it has to be integrated with natural-feeling code workflows.
  • It has to have a full Tier 1 Support model with an SLA.  It can’t be treated the same “Tier 3 Support” way analytics ETL tools are.

Unlike Lambda which is essentially a blank slate for your functions, Decodable is more guided and opinionated.  Decodable aspires to interpret your intentions and get you 90% of the way to having a pipeline built from word go.

Taken together, all of this probably puts Decodable in the same category as things like Kinesis Data Analytics, but KDA requires a bunch of tools (Kinesis, KDA Flink, Glue, Zeppelin) to cover roughly the same surface area, and it isn’t interoperable and it isn’t easy to use. It might include kSQL if you squint, but kSQL isn’t serverless and isn’t interoperable.

This space is new and exciting and we anticipate new market entrants every day.  It will be a lot of fun to watch this category develop as innovations add more requirements to this list!

The Decodable team formed around a common set of problems we saw at customers, and are on a journey to find the easiest way to explain what we've built when it doesn't easily fit any pre-existing categories.

How would you answer the question "What is Decodable?" Drop into the community Slack or message us on Twitter or LinkedIn with your ideas - there may be some swag for the best responses!

You can get started with Decodable for free - our developer account includes enough for you to build a useful pipeline and - unlike a trial - it never expires.

Learn more:

Join the community Slack

‍

📫 Email signup 👇

Did you enjoy this issue of Checkpoint Chronicle? Would you like the next edition delivered directly to your email to read from the comfort of your own home?

Simply enter your email address here and we'll send you the next issue as soon as it's published—and nothing else, we promise!

👍 Got it!
Oops! Something went wrong while submitting the form.
Decodable Team