Streaming Audio: a Confluent podcast about Apache Kafka-logo

Streaming Audio: a Confluent podcast about Apache Kafka

Technology Podcasts >

Streaming Audio is a podcast from Confluent, the team that built Apache Kafka®. Host Tim Berglund (Senior Director of Developer Experience, Confluent) and guests unpack a variety of topics surrounding the Apache Kafka and Confluent ecosystems. The show also features the segment "Ask Confluent," in which Gwen Shapira (Software Engineer, Confluent) and guests respond to a handful of questions and comments from Twitter, YouTube and elsewhere. Apache®, Apache Kafka, Kafka and the flame logo are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. No endorsement by The Apache Software Foundation is implied by the use of these marks.

Streaming Audio is a podcast from Confluent, the team that built Apache Kafka®. Host Tim Berglund (Senior Director of Developer Experience, Confluent) and guests unpack a variety of topics surrounding the Apache Kafka and Confluent ecosystems. The show also features the segment "Ask Confluent," in which Gwen Shapira (Software Engineer, Confluent) and guests respond to a handful of questions and comments from Twitter, YouTube and elsewhere. Apache®, Apache Kafka, Kafka and the flame logo are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. No endorsement by The Apache Software Foundation is implied by the use of these marks.
More Information

Location:

United States

Description:

Streaming Audio is a podcast from Confluent, the team that built Apache Kafka®. Host Tim Berglund (Senior Director of Developer Experience, Confluent) and guests unpack a variety of topics surrounding the Apache Kafka and Confluent ecosystems. The show also features the segment "Ask Confluent," in which Gwen Shapira (Software Engineer, Confluent) and guests respond to a handful of questions and comments from Twitter, YouTube and elsewhere. Apache®, Apache Kafka, Kafka and the flame logo are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. No endorsement by The Apache Software Foundation is implied by the use of these marks.

Language:

English


Episodes

Machine Learning with Kafka Streams, Kafka Connect, and ksqlDB ft. Kai Waehner

12/4/2019
More
In this episode, Kai Waehner (Senior Systems Engineer, Confluent) defines machine learning in depth, describes the architecture of his dream machine learning pipeline, shares about its relevance to Apache Kafka®, Kafka Connect, ksqlDB, and the related ecosystem, and discusses the importance of security and fraud detection. He also covers Kafka use cases, including an example of how Kafka Streams and TensorFlow provide predictive analytics for connected cars. EPISODE LINKS How to Build and...

Duration:00:38:30

Real-Time Payments with Clojure and Apache Kafka ft. Bobby Calderwood

11/27/2019
More
Streamlining banking technology to help smaller banks and credit unions thrive among financial giants is top of mind for Bobby Calderwood (Founder, Evident Systems), who started out in programming, transitioned to banking, and recently launched Evident Real-Time Payments. Payments leverages Confluent Cloud to help banks of all sizes transform to real-time banking services from traditionally batch-oriented, bankers’ hours operational mode. This is achieved through Apache Kafka® and the Kafka...

Duration:00:57:50

Announcing ksqlDB ft. Jay Kreps

11/20/2019
More
Jay Kreps (Co-creator of Apache Kafka® and CEO, Confluent) introduces ksqlDB, an event streaming database. As the successor to KSQL, ksqlDB seeks to unify the multiple systems involved in stream processing into a single, easy-to-use solution for building event streaming applications. ksqlDB offers support for running connectors in an embedded mode, in addition to support for both push and pull queries. Push queries allow you to subscribe to changing query results as new events occur, while...

Duration:00:26:57

Installing Apache Kafka with Ansible ft. Viktor Gamov and Justin Manchester

11/18/2019
More
“It’s one thing to get a distributed system up and running. It’s another thing to get a distributed system up and running well.” Ansible keeps your Apache Kafka® deployment, management, and installation consistent, and it enables you to implement best practices that make it easy to get started. Justin Manchester (Platform DevOps Engineer, Confluent) and Viktor Gamov (Developer Advocate, Confluent) discuss the problems that Ansible is trying to solve, enabling collaboration and optimizing all...

Duration:00:46:06

Securing the Cloud with VPC Peering ft. Daniel LaMotte

11/13/2019
More
Everything is moving to the cloud, which makes it increasingly important to secure your cloud infrastructure and minimize the threat of potential attackers. With a virtual private cloud (VPC)—your own private network in the cloud that you can launch your own instances into—this can be done with VPC Peering, connecting VPCs together to create a path between them to keep your data safe and accessible to you alone. Although typically performed in a single cloud provider, it is possible to do...

Duration:00:31:56

ETL and Event Streaming Explained ft. Stewart Bryson

11/6/2019
More
Migrating from traditional ETL tools to an event streaming platform is a process that Stewart Bryson (CEO and founder, Red Pill Analytics), is no stranger to. In this episode, he dispels misconceptions around what “streaming ETL” means, and explains why event streaming and event-driven architectures compel us to rethink old approaches: Supporting modern, distributed event streaming platforms, and the shift of focus from on-premises to the cloud introduces new use cases that focus primarily...

Duration:00:49:42

The Pro’s Guide to Fully Managed Apache Kafka Services ft. Ricardo Ferreira

11/4/2019
More
Several definitions of a fully managed Apache Kafka® service have floated around, but Ricardo Ferreira (Developer Advocate, Confluent) breaks down what it truly means and why every developer should care. Addressing a handful of questions around Apache Kafka®, Confluent Cloud, hosted solutions, and how they all work, Ricardo describes the benefits of using a fully managed service as a means of simplifying the lives of developers and letting them get back to building—which is why they started...

Duration:00:56:28

Kafka Screams: The Scariest JIRAs and How To Survive Them ft. Anna McDonald

10/30/2019
More
In today's spooktacular episode of Streaming Audio, Anna McDonald (Technical Account Manager, Confluent) discusses six of the scariest Apache Kafka® JIRAs. Starting with KAFKA-6431: Lock Contention in Purgatory, Anna breaks down what purgatory is and how it’s not something to fear or avoid. Next, she dives into KAFKA-8522: Tombstones Can Survive Forever, where she explains tombstones, compacted topics, null values, and log compaction. Not to mention there’s KAFKA-6880: Zombie Replicas Must...

Duration:00:46:32

Data Integration with Apache Kafka and Attunity

10/28/2019
More
From change data capture (CDC) to business development, connecting Apache Kafka® environments, and customer success stories, Graham Hainbach discusses the possibilities of data integration with Kafka and Attunity using Replicate, Compose, and Enterprise Manager. He also shares real-life examples of how Attunity best leverages Kafka in their systems. EPISODE LINKS Apache Kafka Transaction Data Streaming for DummiesJoin the Confluent Community SlackFully managed Apache Kafka as a service!...

Duration:00:43:48

Distributed Systems Engineering with Apache Kafka ft. Colin McCabe

10/23/2019
More
Colin McCabe shares about what it’s like being a distributed systems engineer on the Core Kafka team at Confluent, where he has worked previously, and how that led to his interest in Apache Kafka®. As an active member of the Apache open source community, he describes that the community is a place that both welcomes newcomers and fosters different ideas that help make the product the best that it can be for everyone. Being a distributed systems engineer versus a full stack engineer comes...

Duration:00:45:41

Apache Kafka on Kubernetes, Microsoft Azure, and ZooKeeper with Lena Hall

10/16/2019
More
Lena Hall joins Tim Berglund in the studio to talk about Apache Kafka®, the various ways to run Kafka on Microsoft Azure, Kafka on Kubernetes (K8s), and some exciting events that are happening in the Kafka world. Lena shares about serving double duty as both a senior software engineer and senior cloud developer advocate for Azure Engineering, including her unique roles and responsibilities, and how she balances engineering with advocacy. From writing tech articles to her experience with...

Duration:00:46:08

Improving Fairness Through Connection Throttling in the Cloud with KIP-402 ft. Gwen Shapira

10/9/2019
More
The focus of KIP-402 is to improve fairness in how Apache Kafka® processes connections and how network threads pick up requests and new data. Gwen Shapira (Engineering Manager for Cloud-Native Kafka, Confluent) outlines the details of this KIP and her team’s efforts to make user-facing Kafka improvements. Halfway through the episode, Gwen shares how to send metadata and produce client messages. EPISODE LINKS KIP-402: Improve fairness in SocksetServer processorsJoin the Confluent Community...

Duration:00:48:37

Data Modeling for Apache Kafka – Streams, Topics & More with Dani Traphagen

10/7/2019
More
Helping users be successful when it comes to using Apache Kafka® is a large part of Dani Traphagen’s role as a senior systems engineer at Confluent. Whether she’s advising companies on implementing parts of Kafka or rebuilding their systems entirely from the ground up, Dani is passionate about event-driven architecture and the way streaming data provides real-time insights on business activity. She explains the concept of a stream, topic, key, and stream-table duality, and how each of these...

Duration:00:40:25

MySQL, Cassandra, BigQuery, and Streaming Analytics with Joy Gao

10/2/2019
More
Joy Gao chats with Tim Berglund about all things related to streaming ETL—how it works, its benefits, and the implementation and operational challenges involved. She describes the streaming ETL architecture at WePay from MySQL/Cassandra to BigQuery using Apache Kafka®, Kafka Connect, and Debezium. EPISODE LINKS Cassandra Source Connector DocumentationStreaming Databases in Real Time with MySQL, Debezium, and KafkaStreaming Cassandra at WePayChange Data Capture with Debezium ft. Gunnar...

Duration:00:43:59

Scaling Apache Kafka with Todd Palino

9/25/2019
More
Todd Palino, a senior SRE at LinkedIn, talks about the start of Apache Kafka® at LinkedIn, what learning to use Kafka was like, how Kafka has changed, and what he and others in the community hope for in the future of Kafka. If you’re curious about life as an SRE, Todd shares the details on that too, and goes into how Kafka is used at LinkedIn, as well as several wins and challenges over the years with the product. EPISODE LINKS Kafka: The Definitive Guide by Neha Narkhede, Gwen Shapira &...

Duration:00:46:03

Understand What’s Flying Above You with Kafka Streams ft. Neil Buesing

9/23/2019
More
Neil Buesing (Director of Real-Time Data, Object Partners) discusses what a day in his life looks like and how Kafka Streams helps analyze flight data. EPISODE LINKS Using Location Data to Showcase Keys, Windows, and Joins in Kafka Streams DSL and KSQLKafka: The Definitive Guide by Neha Narkhede, Gwen Shapira & Todd PalinoRead the Confluent blogJoin the Confluent Community Slack

Duration:00:13:00

KIP-500: Apache Kafka Without ZooKeeper ft. Colin McCabe and Jason Gustafson

9/18/2019
More
Tim Berglund sits down with Colin McCabe and Jason Gustafson to talk about KIP-500. The pair, who work on the Kafka Core Engineering Team, discuss the history of Kafka, the creation of KIP-500, and what it will do for the community as a whole. They break down ZooKeeper's role in Kafka, the implications of removing ZooKeeper dependency, replacing it with a self-managed metadata quorum, and how they've been combatting security, stability, and compatibility issues. With pending improvements...

Duration:00:43:46

Should You Run Apache Kafka on Kubernetes? ft. Balthazar Rouberol

9/16/2019
More
When it comes to deploying applications at scale without needing to integrate different pieces of infrastructure yourself, the answer nowadays is increasingly Kubernetes. Kubernetes provides all the building blocks that are needed, and a lot of thought is required to truly create an enterprise-grade Apache Kafka® platform that can be used in production. But before running Kafka on Kubernetes, there are some factors to consider. What are the maturing stages of Kubernetes adoption? How did...

Duration:00:29:37

Jay Kreps on the Last 10 Years of Apache Kafka and Event Streaming

9/12/2019
More
As Confluent turns five years old, special guest Jay Kreps (Co-founder and CEO, Confluent) brings us back to his early development days of coding Apache Kafka® over a Christmas holiday while working at LinkedIn. Kafka has become a breakthrough open source distributed streaming platform based on an abstraction of the distributed commit log, and his involvement in the project eventually led him to start Confluent with Jun Rao and Neha Narkhede. In this episode, Jay shares about all the highs...

Duration:00:48:25

Connecting to Apache Kafka with Neo4j

9/9/2019
More
What’s a graph? How does Cypher work? In today's episode of Streaming Audio, Tim Berglund sits down with Michael Hunger (Lead of Neo4j Labs) and David Allen (Partner Solution Architect, Neo4j) to discuss Neo4j basics and get the scoop on major features introduced in Neo4j 3.4 and 3.5. Among these are geospatial and temporal types, but there’s also more to come in 4.0: a multi-database feature, fine-grained security, and reactive drivers/Spring Data Neo4j RX. In addition to sharing a little...

Duration:00:54:28