The Python Podcast.__init__

Preamble This is a cross-over episode from our new show The Machine Learning Podcast, the show about going from idea to production with machine learning. Summary The majority of machine learning projects that you read about or work on are built around batch processes. The model is trained, and then validated, and then deployed, with each step being a discrete and isolated task. Unfortunately, the real world is rarely static, leading to concept drift and model failures. River is a framework for building streaming machine learning projects that can constantly adapt to new information. In this episode Max Halford explains how the project works, why you might (or might not) want to consider streaming ML, and how to get started building with River. Announcements themachinelearningpodcast.com/deepchecks Interview Contact Info Email@halford_maxMaxHalford Parting Question Closing Announcements Data Engineering PodcastPodcast.__init__sitehosts@themachinelearningpodcast.comiTunes Links Riverscikit-multiflowFederated Machine LearningHogwild!Chip Huyen concept drift blog postDan Crenshaw Berkeley Clipper MLOpsRobustness PrincipleNY Taxi DatasetRiverTorchRiver Public RoadmapBeaverProdigy ML human in the loop labeling The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0

Preamble This is a cross-over episode from our new show The Machine Learning Podcast, the show about going from idea to production with machine learning. Summary Deep learning is a revolutionary category of machine learning that accelerates our ability to build powerful inference models. Along with that power comes a great deal of complexity in determining what neural architectures are best suited to a given task, engineering features, scaling computation, etc. Predibase is building on the successes of the Ludwig framework for declarative deep learning and Horovod for horizontally distributing model training. In this episode CTO and co-founder of Predibase, Travis Addair, explains how they are reducing the burden of model development even further with their managed service for declarative and low-code ML and how they are integrating with the growing ecosystem of solutions for the full ML lifecycle. Announcements pythonpodcast.com/linode Interview Contact Info LinkedIntgaddair@travisaddair Parting Question Closing Announcements Data Engineering PodcastMachine Learning Podcastsitehosts@podcastinit.comiTunes Links PredibaseHorovodLudwigPodcast.__init__ EpisodeSupport Vector MachineHadoopTensorflowUber MichaelangeloAutoMLSpark ML LibDeep LearningPyTorchContinualData Engineering Podcast EpisodeOvertonKubernetesRayNvidia TritonWhylogsData Engineering Podcast EpisodeWeights and BiasesMLFlowCometConfusion MatricesdbtData Engineering Podcast EpisodeTorchscriptSelf-supervised Learning The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0

Preamble This is a cross-over episode from our new show The Machine Learning Podcast, the show about going from idea to production with machine learning. Summary Machine learning has the potential to transform industries and revolutionize business capabilities, but only if the models are reliable and robust. Because of the fundamental probabilistic nature of machine learning techniques it can be challenging to test and validate the generated models. The team at Deepchecks understands the widespread need to easily and repeatably check and verify the outputs of machine learning models and the complexity involved in making it a reality. In this episode Shir Chorev and Philip Tannor explain how they are addressing the problem with their open source deepchecks library and how you can start using it today to build trust in your machine learning applications. Announcements themachinelearningpodcast.com/graftthemachinelearningpodcast.com/predibasethemachinelearningpodcast.com/galileo Interview Contact Info LinkedInshir22LinkedIn@philiptannor Parting Question Closing Announcements Data Engineering PodcastPodcast.__init__sitehosts@themachinelearningpodcast.comiTunes Links DeepChecksRandom ForestTalpiot ProgramSHAPPodcast.__init__ EpisodeAirflowGreat ExpectationsData Engineering Podcast Episode The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0

Preamble This is a cross-over episode from our new show The Machine Learning Podcast, the show about going from idea to production with machine learning. Summary Building an ML model is getting easier than ever, but it is still a challenge to get that model in front of the people that you built it for. Baseten is a platform that helps you quickly generate a full stack application powered by your model. You can easily create a web interface and APIs powered by the model you created, or a pre-trained model from their library. In this episode Tuhin Srivastava, co-founder of Basten, explains how the platform empowers data scientists and ML engineers to get their work in production without having to negotiate for help from their application development colleagues. Announcements pythonpodcast.com/linode Interview Contact Info @tuhinoneLinkedIn Parting Question Closing Announcements Data Engineering PodcastPodcast.__init__sitehosts@themachinelearningpodcast.comiTunes Links BasetenGumroadscikit-learnTensorflowKerasStreamlitPodcast.__init__ EpisodeRetoolHexPodcast.__init__ EpisodeKubernetesReact MonacoHuggingfaceAirtableDall-E 2GPT-3Weights and Biases The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0

Summary Starting a new project is always exciting and full of possibility, until you have to set up all of the repetitive boilerplate. Fortunately there are useful project templates that eliminate that drudgery. PyScaffold goes above and beyond simple template repositories, and gives you a toolkit for different application types that are packed with best practices to make your life easier. In this episode Florian Wilhelm shares the story behind PyScaffold, how the templates are designed to reduce friction when getting a new project off the ground, and how you can extend it to suit your needs. Stop wasting time with boring boilerplate and get straight to the fun part with PyScaffold! Announcements pythonpodcast.com/linode Interview Keep In Touch WebsiteLinkedInFlorianWilhelm@florianwilhelm Picks DaredevilThe Peripheral Closing Announcements Data Engineering PodcastMachine Learning Podcastsitehosts@podcastinit.comiTunes Links PyScaffoldInnovexSAPCookiecutterPytestPodcast EpisodeSphinxpre-commitPodcast EpisodeBlackFlake8Podcast EpisodePoetrySetuptoolsmkdocsReStructured TextMarkdownSetuptools-SCMHatchFlitVersioneerGourceMyPy CompilerRust Cargo The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Summary Application configuration is a deceptively complex problem. Everyone who is building a project that gets used more than once will end up needing to add configuration to control aspects of behavior or manage connections to other systems and services. At first glance it seems simple, but can quickly become unwieldy. Bruno Rocha created Dynaconf in an effort to provide a simple interface with powerful capabilities for managing settings across environments with a set of strong opinions. In this episode he shares the story behind the project, how its design allows for adapting to various applications, and how you can start using it today for your own projects. Announcements pythonpodcast.com/linode Interview Keep In Touch rochacbruno@rochacbrunoWebsiteLinkedIn Picks SOPSSeveranceRust Closing Announcements Data Engineering PodcastMachine Learning Podcastsitehosts@podcastinit.comiTunes Links DynaconfDynaconf GitHub OrgAnsibleBashPerl12 Factor ApplicationsTOMLHashicorp VaultPydanticAirflowHydroconf The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Summary Software is eating the world, but that code has to have hardware to execute the instructions. Most people, and many software engineers, don’t have a proper understanding of how that hardware functions. Charles Petzold wrote the book "Code: The Hidden Language of Computer Hardware and Software" to make this a less opaque subject. In this episode he discusses what motivated him to revise that work in the second edition and the additional details that he packed in to explore the functioning of the CPU. Announcements pythonpodcast.com/linode Interview Keep In Touch Website Picks The Imitation GameThe Annotated TuringConfidence Man: The Making of Donald Trump and the Breaking of America Links Code: The Hidden Language of Computer Hardware and SoftwareFortranPL/IBASICC#Z80Intel 8080PC MagazineAssembly LanguageLogic GatesC LanguageASCII == American Standard Code for Information InterchangeSkiaSharpAlgolCode first edition bibliography The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Summary The generation, distribution, and consumption of energy is one of the most critical pieces of infrastructure for the modern world. With the rise of renewable energy there is an accompanying need for systems that can respond in real-time to the availability and demand for electricity. FlexMeasures is an open source energy management system that is designed to integrate a variety of inputs intelligently allocate energy resources to reduce waste in your home or grid. In this episode Nicolas Höning explains how the project is implemented, how it is being used in his startup Seita, and how you can try it out for your own energy needs. Announcements pythonpodcast.com/linode Interview Keep In Touch Website@nhoeningLinkedIn Picks She-HulkKleoAltair Closing Announcements Data Engineering PodcastMachine Learning Podcastsitehosts@podcastinit.comiTunes Links GithubLinux Energy FoundationMailing ListTwitterEyeQuantEnergy Management SystemOpenEMSICT == Information and Communications TechnologyHomeAssistantPodcast EpisodeFlexMeasures HomeAssistant PluginUniversal Smart Energy FrameworkPostgreSQLData Engineering Podcast EpisodeTimescaleDBData Engineering Podcast EpisodeOpenWeatherMapTimely-BeliefsFlaskClickPyomoscikit-learnsktimeLF EnergyFlake8MyPyPodcast EpisodeBlackArima ModelRandom Forest The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Summary Your ability to build and maintain a software project is tempered by the strength of the team that you are working with. If you are in a position of leadership, then you are responsible for the growth and maintenance of that team. In this episode Jigar Desai, currently the SVP of engineering at Sisu Data, shares his experience as an engineering leader over the past several years and the useful insights he has gained into how to build effective engineering teams. Announcements pythonpodcast.com/linodepythonpodcast.com/selectstar Interview Keep In Touch LinkedIn Picks Bullet TrainTop Gun Maverick Closing Announcements Data Engineering PodcastMachine Learning Podcastsitehosts@podcastinit.comiTunes Links Sisu DataOpenStackJava The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Summary Working on hardware projects often has significant friction involved when compared to pure software. Brian Pugh enjoys tinkering with microcontrollers, but his "weekend projects" often took longer than a weekend to complete, so he created Belay. In this episode he explains how Belay simplifies the interactions involved in developing for MicroPython boards and how you can use it to speed up your own experimentation. Announcements pythonpodcast.com/linodepythonpodcast.com/selectstar Interview Keep In Touch BrianPughLinkedIn Picks Gunnar Computer Glasses Closing Announcements Data Engineering PodcastMachine Learning Podcastsitehosts@podcastinit.comiTunes Links BelayGeomagicalPIC MicrocontrollerAVR MicrocontrollerMatlabMicroPythonPodcast EpisodeCircuitPythonPodcast EpisodeCeleryPotentiometerRaspberry PiRaspberry Pi PicoADC ConverterThonnyPodcast EpisodeAdafruitPyboardPython Inspect ModulePython TokenizeMagnetometer ProjectLidar The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Summary Static typing versus dynamic typing is one of the oldest debates in software development. In recent years a number of dynamic languages have worked toward a middle ground by adding support for type hints. Python’s type annotations have given rise to an ecosystem of tools that use that type information to validate the correctness of programs and help identify potential bugs. At Instagram they created the Pyre project with a focus on speed to allow for scaling to huge Python projects. In this episode Shannon Zhu discusses how it is implemented, how to use it in your development process, and how it compares to other type checkers in the Python ecosystem. Announcements pythonpodcast.com/linode Interview Keep In Touch shannonzhu Picks Lord Of The Rings: The Rings of PowerKing’s Dilemma Closing Announcements Data Engineering PodcastMachine Learning Podcastsitehosts@podcastinit.comiTunes Links PYreMyPyPodcast EpisodePyRightPyTypeMonkeyTypePodcast EpisodeJavaCPEP 484FlowHackContinuous IntegrationOCamlPEP 675Gradual TypingAST == Abstract Syntax TreeLanguage Server ProtocolTensorType ArithmeticPyCon: Securing Code With The Python Type SystemPyCon: Type Checked Python In The Real WorldPyCon: Łukasz Lange 2022 Keynote The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Summary Every software project is subject to a series of decisions and tradeoffs. One of the first decisions to make is which programming language to use. For companies where their product is software, this is a decision that can have significant impact on their overall success. In this episode Sean Knapp discusses the languages that his team at Ascend use for building a service that powers complex and business critical data workflows. He also explains his motivation to standardize on Python for all layers of their system to improve developer productivity. Announcements pythonpodcast.com/linode Interview Keep In Touch LinkedIn@seanknapp Picks Delver LensMagic: The GatheringTyperDuckDBAmp It Up Closing Announcements Data Engineering PodcastMachine Learning Podcastsitehosts@podcastinit.comiTunes Links AscendData Engineering Podcast EpisodePerlGoogle SawzallTechnical DebtRubygRPCGo LanguageJavaPySparkApache ArrowThriftSQLScalaSnowflakeSnowparkTyperPydanticPodcast EpisodePulumiPodcast EpisodePyInfraPodcast EpisodePackerPlot.ly DashDuckDB The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Summary Writing code is only one piece of creating good software. Code reviews are an important step in the process of building applications that are maintainable and sustainable. In this episode On Freund shares his thoughts on the myriad purposes that code reviews serve, as well as exploring some of the patterns and anti-patterns that grow up around a seemingly simple process. Announcements pythonpodcast.com/linode Interview Keep In Touch LinkedIn@onfreund Picks The Girl Who Drank The MoonBetter Call Saul Closing Announcements Data Engineering PodcastMachine Learning Podcastsitehosts@podcastinit.comiTunes Links WilcoCode ReviewHome AssistantPodcast EpisodeTrunk-based DevelopmentGit FlowPair ProgrammingFeature FlagsPodcast EpisodeKPI == Key Performance IndicatorMIT Open Learning Engineering HandbookPEP Repository The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Summary Quality assurance in the software industry has become a shared responsibility in most organizations. Given the rapid pace of development and delivery it can be challenging to ensure that your application is still working the way it’s supposed to with each release. In this episode Jonathon Wright discusses the role of quality assurance in modern software teams and how automation can help. Announcements pythonpodcast.com/linode Interview Keep In Touch LinkedIn@Jonathon_WrightWebsite Picks Netflix seriesGraphic NovelsNeil GaimainHouse of the DragonMystic QuestIt’s Always Sunny in Philadelphia Links HaskellIdrisEsperantoKlingonPlanguageLisp LanguageTDD == Test Driven DevelopmentBDD == Behavior Driven DevelopmentGherkin FormatIntegration TestingChaos EngineeringGremlinChaos ToolkitPodcast EpisodeRequirements EngineeringKeysightQA Lead PodcastCognitive Learning TED TalkOpenTelemetryPodcast EpisodeQuality EngineeringSeleniumSwaggerXPathRegular ExpressionTest Guild The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Summary The goal of every software team is to get their code into production without breaking anything. This requires establishing a repeatable process that doesn’t introduce unnecessary roadblocks and friction. In this episode Ronak Rahman discusses the challenges that development teams encounter when trying to build and maintain velocity in their work, the role that access to infrastructure plays in that process, and how to build automation and guardrails for everyone to take part in the delivery process. Announcements pythonpodcast.com/linode Interview Keep In Touch @OfRonak Picks The Terminal ListMidnight Gospel Closing Announcements Data Engineering PodcastMachine Learning Podcastsitehosts@podcastinit.comiTunes Links QualiTorqueVisual Studio PluginSubversionIaC == Infrastructure as CodeDevOpsTerraformPulumiPodcast EpisodeCloudformationFlask The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Summary Every startup begins with an idea, but that won’t get you very far without testing the feasibility of that idea. A common practice is to build a Minimum Viable Product (MVP) that addresses the problem that you are trying to solve and working with early customers as they engage with that MVP. In this episode Tony Pavlovych shares his thoughts on Python’s strengths when building and launching that MVP and some of the potential pitfalls that businesses can run into on that path. Announcements pythonpodcast.com/linode Interview Keep In Touch LinkedIn Picks datamodel-code-generatorScrew It, Let’s Do It Closing Announcements Data Engineering PodcastMachine Learning Podcastsitehosts@podcastinit.comiTunes Links PLANEKSMinimum Viable ProductDjangoCookiecutterDjango BoilerplateOCR == Optical Character RecognitionTesseract The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Summary Application architectures have been in a constant state of evolution as new infrastructure capabilities are introduced. Virtualization, cloud, containers, mobile, and now web assembly have each introduced new options for how to build and deploy software. Recognizing the transformative potential of web assembly, Matt Butcher and his team at Fermyon are investing in tooling and services to improve the developer experience. In this episode he explains the opportunity that web assembly offers to all language communities, what they are building to power lightweight server-side microservices, and how Python developers can get started building and contributing to this nascent ecosystem. Announcements pythonpodcast.com/linodedataengineeringpodcast.com/shipyard Interview Keep In Touch LinkedIn@technosophostechnosophos Picks Thor: Love & ThunderRemembrance of Earth’s Past Closing Announcements Data Engineering PodcastMachine Learning Podcastsitehosts@podcastinit.comiTunes Links FermyonPython entryWasm Language MatrixSingleStore’s WASI-PythonWasm support in CPytonPyodideSlashDotWeb Assembly (WASM)RustAssemblyScriptGrainSingleStoreData Engineering Podcast EpisodeWASIPyO3PyOxidizerRustPythonDrupalOpenStackDeisHelmRedPandaData Engineering Podcast EpisodeEnvoy ProxyFastlyFunctions as a ServiceCloudEventsFinicky WhiskersFermyon SpinNomadTree ShakingZappaChaliceOpenFaaSCNCFBytecode AllianceFinicky Whiskers MinecraftKotlin The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Summary As your code scales beyond a trivial level of complexity and sophistication it becomes difficult or impossible to know everything that it is doing. The flow of logic and data through your software and which parts are taking the most time are impossible to understand without help from your tools. VizTracer is the tool that you will turn to when you need to know all of the execution paths that are being exercised and which of those paths are the most expensive. In this episode Tian Gao explains why he created VizTracer and how you can use it to gain a deeper familiarity with the code that you are responsible for maintaining. Announcements pythonpodcast.com/linodedataengineeringpodcast.com/shipyard Interview Keep In Touch gaogaotiantianLinkedIn Picks TravelersobjprintLincoln Lawyerbilibili Closing Announcements Data Engineering PodcastMachine Learning Podcastsitehosts@podcastinit.comiTunes Links ViztracerPython cProfileSampling ProfilerPerfettoCoverage.pyPodcast EpisodePython setxprofile hookCircular BufferCatapult Trace Viewerpy-spypsutilgdbFlame graph The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Summary Analysis of streaming data in real time has long been the domain of big data frameworks, predominantly written in Java. In order to take advantage of those capabilities from Python requires using client libraries that suffer from impedance mis-matches that make the work harder than necessary. Bytewax is a new open source platform for writing stream processing applications in pure Python that don’t have to be translated into foreign idioms. In this episode Bytewax founder Zander Matheson explains how the system works and how to get started with it today. Announcements pythonpodcast.com/linodepythonpodcast.com/selectstardataengineeringpodcast.com/shipyard Interview Keep In Touch SlackTwitterLinkedIn Picks Alta RacksAtherton Bikes Links BytewaxGitHubFlinkData Engineering Podcast EpisodeSpark StreamingKafka ConnectFaustPodcast EpisodeRayPodcast EpisodeDaskData Engineering Podcast EpisodeTimely DataflowPyO3MaterializeData Engineering Podcast EpisodeHyperLogLogPython River LibraryShannon Entropy CalculationThe blog post using incremental shannon entropyNATSwaxctlPrometheusGrafanaStreamz The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Summary Building a fully functional web application has been growing in complexity along with the growing popularity of javascript UI frameworks such as React, Vue, Angular, etc. Users have grown to expect interactive experiences with dynamic page updates, which leads to duplicated business logic and complex API contracts between the server-side application and the Javascript front-end. To reduce the friction involved in writing and maintaining a full application Sam Willis created Tetra, a framework built on top of Django that embeds the Javascript logic into the Python context where it is used. In this episode he explains his design goals for the project, how it has helped him build applications more rapidly, and how you can start using it to build your own projects today. Announcements pythonpodcast.com/linodepythonpodcast.com/selectstardataengineeringpodcast.com/shipyard Interview Keep In Touch @samwillisWebsiteLinkedInsamwillis Picks The Machine Learning PodcastSlow Horses Closing Announcements Data Engineering PodcastMachine Learning Podcastsitehosts@podcastinit.comiTunes Links Tetra FrameworkDjangoPHPASPAlpine.jsHTMXRubyRuby on RailsFlutterboxVue.jsLaravel LivewirePython Import Hookspython-inline-sourceTailwind CSSPostCSSPickleFernetesbuildWebpackRich The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA