apply() Conference 2022 | Streaming is just an Implementation Detail

Materialize

August 5, 2022

About the Video

Microservices are stream processing; whether you’re using Redis, Kafka, or gRPC, you continuously handle events and manage consistency. And given that these are some of the most challenging problems in databases, you’re probably not doing a very good job at it.

But that’s not your fault, these problems are hard! Just like you wouldn’t implement your own database for every web service, you shouldn’t be building your own stream processor for every new product feature.

Today’s stream processors have failed to gain widespread adoption outside niche use cases because they put themselves at the forefront. They force you to think about streaming when building your application and when deploying to production.

In this talk Arjun argues “streaming” is the right tool but the wrong abstraction. Of course, we all want the benefits – increased speed, stronger consistency – but they need to meet developers where they are. Only when streaming becomes an implementation detail can it gain widespread adoption and bring forth the benefits it has promised for so long.