Software Misadventures

If you’ve worked on data problems, you probably have heard of Airflow and Superset, two powerful tools that have cemented their place in the data ecosystem. Building successful open-source software is no easy feat, and even fewer engineers have done this back to back. In part 2 of the conversation, we talk about Max’s journey in open source. Segments: (00:03:27) “Project-Community Fit” in Open Source (00:08:31) Fostering Relationships in Open Source (00:10:58) Dealing with Trolls (00:13:40) Attributes of Good Open Source Contributors (00:20:01) How to Get Started with Contributing (00:27:58) Origin Stories of Airflow and Superset (00:33:27) Biggest Surprise since Founding a VC-backed Company? (00:38:47) Picking What to Work On (00:41:46) Advice to Engineers for Building the Next Airflow/Superset? (00:42:35) The 2 New Open Source Projects that Max is Starting (00:52:10) Challenges of Being a Founder (00:57:38) Open Sourcing Ideas Show Notes: Part 1 of our conversation: https://softwaremisadventures.com/p/maxime-beauchemin-llm-ready Max on LinkedIn: https://www.linkedin.com/in/maximebeauchemin/ SQL All Stars: https://github.com/preset-io/allstars Governator: https://github.com/mistercrunch/governator Stay in touch: 👋 Make Ronak’s day by leaving us a review and let us know who we should talk to next! hello@softwaremisadventures.com

If you’ve worked on data problems, you probably have heard of Airflow and Superset, two powerful tools that have cemented their place in the data ecosystem. Building successful open-source software is no easy feat, and even fewer engineers have done this back to back. In Part 1 of this conversation, we chat about how to adapt to the LLM-age as engineers. Segments: (00:01:59) The Rise and Fall of the Data Engineer (00:11:13) The Importance of Executive Skill in the Era of AI (00:13:53) Developing the first reflex to use AI (00:17:47) What are LLMs good at? (00:25:33) Text to SQL (00:28:19) Promptimize (00:32:16) Using tools LangChain (00:35:02) Writing better prompts Show Notes: - Max on Linkedin: https://www.linkedin.com/in/maximebeauchemin/ - Rise of the Data Engineer: https://medium.com/free-code-camp/the-rise-of-the-data-engineer-91be18f1e603 - Downfall of the Data Engineer: https://maximebeauchemin.medium.com/the-downfall-of-the-data-engineer-5bfb701e5d6b - Promptimize: https://github.com/preset-io/promptimize Stay in touch: 👋 Make Ronak’s day by leaving us a review and let us know who we should talk to next! hello@softwaremisadventures.com

Out of thousands of engineers at Uber, there’s only a handful of Distinguished Engineers and Joakim was one of them. In this conversation we chat about Why software engineering is a lot like a sausage factory. Considerations for leaving big tech for a startup. “How to beat the promo commitee”. How can one effectively shape engineering culture? “Mentoring two people on the same team is a waste”. Much More. Subscribe now Segments: [0:01:52] The “reverse sausage” architecture [0:07:36] How to get people on board with the new deployment system? [0:13:55] What does it mean to be a distinguished engineer? [0:17:47] Under-appreciated soft skills? [0:21:28] How to improve technical writing [0:24:16] Do all senior engineers need to write and review code every day? [0:30:19] How to search out where to contribute when your time is so constrained? [0:43:10] How to maximize your impact as a mentor [0:48:52] “How to beat the promo committee” [0:52:56] Effective means to influence engineering culture? [0:57:09] Capping the company at 150 employees [1:03:33] Why join a startup instead of moving to another big tech company? [1:11:14] What Joakim is working on now at Beyond Work Show Notes: Joakim on leaving Uber to start Beyond Work: https://www.linkedin.com/pulse/why-i-left-uber-start-beyond-work-joakim-recht-o63of?trk=public_post_feed-article-content Read Joakim’s other excellent posts here: https://www.linkedin.com/in/recht/recent-activity/all/ Stay in touch: 👋 Make Ronak’s day by leaving us a review and let us know who we should talk to next! hello@softwaremisadventures.com

We’re super excited to have Kelsey back on the show! Our last conversation was around his incredible career journey - from working at McDonald’s after school to starting his own computer store, to hacking on python infrastructure with the core developers, to meeting Satya Nadella for an interview. In part two of this conversation, we dive deep into Kelsey’s experiences learning in public and writing “Kubernetes: Up and Running”: The biggest barrier to getting started with learning in public and a step-by-step guide to overcome it Cautionary tale of the “JavaScript sucks” guy Developing the skill of crafting good analogies The business and economics of writing a book Much more Segments: [0:01:12] Writing and learning in public. [0:10:58] Writing "Kubernetes: Up and Running." [0:16:05] The business and economics of writing a book. [0:21:27] Why your first book should not exceed 100 pages. [0:23:36] What prevented Kelsey from giving up on the book. [0:26:15] Being intentional about building an audience and the cautionary tale of the "JavaScript sucks" guy. [0:36:44] Authenticity does not guarantee success. [0:39:09] Developing the skill of crafting effective analogies. [0:47:47] Advice for engineers to leverage their technical skills outside of the nine-to-five. Show Notes: Kelsey on twitter: https://twitter.com/kelseyhightower Our previous conversation with Kelsey about retiring as Distinguished Engineer from Google at 42: https://softwaremisadventures.com/p/kelsey-hightower-on-retiring-as-distinguished-057 Stay in touch: 👋 Make Ronak’s day by leaving us a review and let us know who we should talk to next! hello@softwaremisadventures.com

We’re super excited to have Kelsey back on the show! Our last conversation was around his incredible career journey - from working at McDonald’s after school to starting his own computer store, to hacking on python infrastructure with the core developers, to meeting Satya Nadella for an interview. In part one of this conversation, we dive deep into Kelsey’s experiences and expertise as a startup advisor: How to break into advising when you don’t have a lot of connections How to influence without authority Passive vs. active advising How to add value as an advisor Setting boundaries and expectations Much more Segments: [0:01:53] Being a "junior retiree" [0:11:00] How Kelsey got started with startup advising. [0:17:43] How to avoid mismatches in advisory engagements? [0:27:23] How to influence without authority as an advisor? [0:32:58] How to establish boundaries as an advisor. [0:38:29] Actions engineers can take today to prepare themselves for future startup advising roles. [0:42:55] How to manage the balance between advising and your primary job. [0:44:32] How to cultivate perspectives beyond engineering. Show Notes: Kelsey on twitter: https://twitter.com/kelseyhightower Our previous conversation with Kelsey about retiring as Distinguished Engineer from Google at 42: https://softwaremisadventures.com/p/kelsey-hightower-on-retiring-as-distinguished-057 Stay in touch: 👋 Make Ronak’s day by leaving us a review and let us know who we should talk to next! hello@softwaremisadventures.com

As a self-described “gainfully unemployed data person”, Josh Wills is an angel investor and has worked on and led data teams at Slack, Cloudera, WeaveGrid and Google. We discuss: How to get started with angel investing without a ton of $$ Attributes that define great engineering managers What’s it like transitioning from management back to IC Challenges in Climate Tech from a software perspective And more Segments: [0:01:35] Transitioning from management to individual contributor (IC). [0:10:19] Emotional intelligence and its role in engineering management. [0:25:21] Contrasting the hard power of management with the soft power of senior individual contributors. [0:37:18] Addressing challenges in climate technology. [0:51:34] The importance of practicality and how to assess it in interviews. [0:56:01] Josh's journey into angel investing. [1:12:59] Criteria used by Josh to evaluate whether to invest in a startup. Show Notes: Josh on Twitter: https://twitter.com/josh_wills The “Touchy Feely” course at Stanford: https://www.gsb.stanford.edu/experience/learning/leadership/interpersonal-dynamics Jason Calacanis’s book on angel investing: https://www.amazon.com/Angel-Invest-Technology-Startups-Timeless-Investor/dp/0062560700 Stay in touch: 👋 Make Ronak’s day by leaving us a review and let us know who we should talk to next! hello@softwaremisadventures.com

Known for coining the term “Data Scientist”, DJ is a renowned technologist with a diverse background spanning academia, industry, and government. Having led product teams at companies like RelateIQ and LinkedIn, DJ was appointed by President Obama to be the first U.S. Chief Data Scientist where his efforts led to the establishment of nearly 40 Chief Data Officer roles across the Federal government, new health care programs as well as new criminal justice reforms. We discuss: “Dream in years, plan in months, evaluate in weeks, ship daily” High school misadventures that shaped DJ’s world view Under-hyped opportunities in AI Building with the customer vs. “if you build it, they will come” Do we need more regulations on AI? Much more. Segments: [0:01:48] Picking locks in high school. [0:07:15] How can we make it easier for others to take a risk on us? [0:11:29] How do you decide whom to take a chance on? [0:14:24] The 70-20-10 framework for choosing what to work on. [0:17:49] "No rules, only guidelines." [0:24:09] Developing personal ethics. [0:30:52] Building with the customer versus "if you build it, they will come." [0:34:51] "Dream in years, plan in months, evaluate in weeks, ship daily." [0:43:56] Ideas should be considered in terms of momentum. [0:46:11] Under-hyped trends in AI? [0:51:53] How does AI need to evolve to operate in fields that require very low margins of error? [0:56:09] Concerning advances that lack sufficient guardrails? [0:58:55] Do we need more regulations on AI? [1:02:48] "Failure is the only option." Show Notes: DJ Patil on Linkedin: https://www.linkedin.com/in/dpatil/ The card that DJ carried in his notebook: https://twitter.com/DJ44/status/819316928623902720 DJ’s interview series with thought leaders in Data Science: https://www.linkedin.com/learning/data-impact-with-dj-patil/data-science-how-did-we-get-here Stay in touch: 👋 Make Ronak’s day by leaving us a review and let us know who we should talk to next! hello@softwaremisadventures.com

Erica is a former VP of Engineering at LinkedIn. Having almost dropped out of college, Erica’s journey in tech is a testament to her perseverance and dedication. In addition to leading engineering teams at LinkedIn, Erica founded WIT (Women In Tech) to empower women within the company as well as the broader tech community. We discuss: How to create incentives for diversity-building work. Building your personal “board of directors”. Balancing mentoring work vs sprint tickets. Structuring a community for long-term success. Much more. Segments: [0:18:04] building women-in-tech and the importance of leading by example [0:21:17] creating incentives for diversity-building work [0:23:30] examples of building better products with more diverse stakeholders [0:29:48] how to spot red flags during the interview process [0:32:51] do men and women bring different skill sets to the problem or it's all individual based? [0:35:34] building your personal “board of directors” [0:40:21] how to ask people for mentorship if I’m shy? [0:44:21] exploring new projects [0:53:32] how to hold yourself accountable when there’s no structure? [1:03:17] how to structure a community for long-term success [1:10:22] how to balance mentoring work vs sprint tickets [1:14:57] journey to being on the advisory board for SJU Show Notes: Erica on Linkedin: https://www.linkedin.com/in/ericalockheimer/ Stay in touch: 👋 Make Ronak’s day by leaving us a review and let us know who we should talk to next! hello@softwaremisadventures.com

Mitchell co-founded HashiCorp in 2012 and created many important infrastructure tools, such as Terraform, Vagrant, Packer, and Consul. In addition to being a prolific engineer, Mitchell grew HashiCorp into a multi-billion-dollar public company. We discuss: How to structure large projects to avoid demotivation or burnout The "A.P.P.L.E" framework for diffusing tense situations and handling trolls How to decide what to work on Mitchell's unconventional transitions from CEO to CTO and then back to an individual contributor (IC) The quality that Mitchell values the most in an engineering team. Segments: [0:14:19] Impactful lessons from working at the Apple Store in college [0:22:26] Origin story of HashiCorp [0:26:08] College side project that turned into Mitchell’s first financial success [0:31:25] Why infrastructure? [0:39:50] How individual products came about [0:44:17] Challenges of fundraising as a company with an umbrella of products [0:48:20] Balancing being the CTO and writing code: “I didn’t want to be that CTO that just produced technical debt” [0:53:09] Transitioning from CEO to co-CTO [0:57:26] From CTO to Individual Contributor [1:06:03] What’s next? Show Notes: Mitchell’s blog: https://mitchellh.com/writing The “APPLE” principle that has guided Mitchell throughout his career: https://mitchellh.com/writing/apple-the-key-to-my-success Mitchell’s Startup Banking Story 😂: https://mitchellh.com/writing/my-startup-banking-story Stay in touch: 👋 Make Ronak’s day by leaving us a review and let us know who we should talk to next! hello@softwaremisadventures.com

After 17 years building SRE teams at Google and serving as the Site Lead for Engineering in Dublin, Dave joined Elastic as the Sr Director of Engineering and later VP of Engineering at Twilio. Following a recent career break, Dave now divides his time between coaching engineering leaders and consulting to help busy teams be more effective. In the heart of our conversation, Dave shares the frameworks and practical tips he's amassed for making the most of the mentorship experience. Segments: [00:01:45] Growing remote SRE team as the Google Dublin Site Lead [00:19:49] Company Culture vs Company Values [00:23:47] How to find companies that are serious about remote work [00:34:26] Coaching vs Mentoring at Big vs Small companies [00:38:35] How Google does coaching & mentoring [00:41:38] What makes a good 1-1 [00:46:56] Considerations for seeking out a mentor [01:03:27] Getting external mentorship while working at a small company [1:08:20] How to set specific goals for mentorship [1:20:13] The “CIA” Method for career decision making [1:31:08] How to sunset mentorship 1-1s [1:35:20] Venturing into consulting to help busy teams be more effective [1:42:13] How to get started with consulting Show Notes: Dave on LinkedIn: https://www.linkedin.com/in/gerrowadat/ Dave’s personal website: https://log.andvari.net/pages/about.html Dave’s coaching website: https://www.strategichopes.co/ Service Level Objectives by Alex Hidalgo: https://www.oreilly.com/library/view/implementing-service-level/9781492076803/ The Staff Engineer’s Path by Tanya Reilly: https://www.oreilly.com/library/view/the-staff-engineers/9781098118723/ Stay in touch: 👋 Let us know who we should talk to next! hello@softwaremisadventures.com

At the personal request of Reid Hoffman to emerge from early retirement, David joined LinkedIn in 2009 during a period of rapid growth to help stabilize the chaos, cultivating a much-needed culture of “Site Up and Secure.” Before this, David served as SVP of Engineering and Operations at Yahoo!, overseeing their Search Marketing organization and the Production Operations infrastructure for the entire company. Throughout his career, David has held multiple leadership positions and is recognized as one of the top operations executives. David’s intensity, passion, courage and commitment to work have always been deeply admired by his colleagues and his wisdom, well captured in one line axioms, better known as Henkeisms, are still echoed at LinkedIn. This episode was first published almost 3 years ago and we are sharing it again because it’s been one of our favorites :) Hope you like it too! Segments: [00:01:37] “This is my freaking site” poster [00:04:10] David’s first 2 retirements and starting at LinkedIn [00:09:41] IC to Management [00:17:20] Site-Up Culture [00:21:58] Re-architecting LinkedIn’s release process [00:27:23] War stories from Yahoo: The 10G Massacre [00:32:06] “Go to work every day willing to be fired”: Project Panama at Yahoo [00:43:33] The power of Individual Contributors Show Notes: David Henke on LinkedIn Learning to Lead - David Henke’s talk on leadership which he delivered at his alma mater - UCSB Project Inversion at LinkedIn Stay in touch: 👋 Let us know who we should talk to next! hello@softwaremisadventures.com

Before joining CueIn last year as a Founding Data Scientist, Melissa was a Lead Data Scientist at Salesforce working on the Einstein Platform that focused on automating Data Science workflows. In this conversation we dive into Melissa’s unique journey, what to do in the face of increasing job automation and explore the latest developments in practical AI. Segments: [00:02:13] Melissa’s background in computational neuroscience [00:06:08] 7 years at Salesforce vs startup [00:11:31] Joining CueIn [00:19:30] Chatbot observability [00:28:16] Feedback loops [00:33:10] Use LLM to observe.. LLMs? [00:39:06] AI automating jobs [00:43:01] Doing ML in 2017 vs now [00:50:35] Few shot learning, Hugging Face Show Notes: Melissa’s Linkedin: https://www.linkedin.com/in/melissajanerunfeldt/ Stay in touch: 👋 Let us know who we should talk to next! hello@softwaremisadventures.com

What's it like to open source an internal project at a big tech company like LinkedIn? When should a company open source a project and what are the benefits and challenges that come along with it? If you want to open source an internal project, how should you go about advocating for it? Félix is a Principal Staff Engineer at LinkedIn where he works on the data infrastructure team that builds Venice. Venice is a distributed derived data store which LinkedIn open sourced in the fall of 2022. He joins the show to chat about his experiences leading the open source efforts for Venice, as well as his thoughts on balancing leadership with execution, delegating responsibility and fostering a culture of ownership, and growth within a team. --- Show Notes: https://github.com/linkedin/venicehttps://github.com/linkedin/venice --- Stay in Touch: ✉️ Subscribe to our newsletter: https://softwaremisadventures.com 👋 Let us know who we should talk to next! hello@softwaremisadventures.com --- Segments: [0:01:36] Introduction [0:02:32] Career Choices and Job Satisfaction [0:08:34] Understanding Venice: LinkedIn's Distributed Derived Data Store [0:22:37] The Journey of Open-Sourcing Venice [0:26:36] Understanding the Business Perspective of Open Source Systems [0:30:28] How and when to advocate for open-sourcing an internal project [0:39:32] Challenges and Strategies in Open Source Project Maintenance [0:46:40] Balancing Leadership and Execution in Engineering Roles

Should engineers and product managers “stay in their lanes”? What big company habits should you keep vs unlearn when transitioning to working at a start-up? Could an ayahuasca retreat give you more clarity on your career goals? Ilya and Arnab join the show to share their journey quitting big tech to bootstrap a podcasting startup. Arnab and Ilya are the co-founders of Metacast. Before starting the company, Arnab was a Principal Engineer at AWS while Ilya was a Sr. Product Manager at Google and Principal PM at Amazon before that. While at Amazon, Arnab and Ilya worked together on various projects including AWS Chatbot, which they started from scratch and launched into a successful AWS service. Show Notes: metacast.apphttps://www.metacastpodcast.com/ Stay in Touch: ✉️ Subscribe to our newsletter: https://softwaremisadventures.com 👋 Let us know who we should talk to next! hello@softwaremisadventures.com Segments: [0:00:00] Starting Metacast [0:05:39] Should engineers and product managers “stay in their lanes”? [0:11:56] How to decide when to explore options vs committing to a decision [0:14:46] Do you have to be a specialist to be successful? [0:18:20] Quitting Amazon & Google [0:33:52] Spiritual retreat [0:47:09] Trying therapy [0:51:33] Orthogonal weaknesses [0:57:31] Big co habits to keep vs unlearn [1:04:32] Metacast Milestones

What's "AI in a Box"? Pete Warden joins the show to share a new project he recently launched that encapulates Language Transcription/Translation and Question Answering capabilities into a wallet-sized board running locally without internet, as well as stories and learnings from building his new company, Useful Sensors, after 7 years of leading the tensorflow mobile project at Google. Pete is the CEO of Useful Sensors. After founding his own company Jetpac and selling it to Google in 2014, he became a staff research engineer at Google, where he led the TensorFlow Mobile team. Pete is also the author of two well-received books: "Public Data Sources" and "Big Data Glossary" and builder of OpenHeatMap. Show Notes: https://petewarden.com/https://usefulsensors.com/ Stay in touch: ✉️ Subscribe to our newsletter: https://softwaremisadventures.com 👋 Send feedback or say hi: hello@softwaremisadventures.com Segments: [0:00:00] Failing and trying again, experiential learning [0:03:13] AI-in-a-box demo [0:07:28] Animatronics? [0:10:12] Privacy and trust [0:12:04] Talk to your appliances [0:15:22] How to fit the LLM into such a small chip? [0:16:50] Quantization [0:20:07] Disposable ML frameworks [0:24:33] Updating model on shipped hardware [0:28:34] LLMs with specialized domain knowledge [0:30:08] Founding Useful Sensors [0:37:07] scaling training vs inference

What does it mean to change the economics of software development? Nathan Marz joins the show to share how they reduced the cost of building Mastodon at Twitter-scale by 100X and the 10 years journey to build Rama, a new programming platform that made this feat possible. Nathan is the founder of Red Planet Labs. Prior to RPL, he led engineering for BackType which was acquired by Twitter in 2011. Nathan created the Apache Storm project and wrote the book Big Data: Principles and best practices of scalable realtime data systems. Outside of working, Nathan is a private pilot, loves going to stand-up comedy shows, and is forever trying to teach his dog new tricks. Show Notes: https://twitter.com/nathanmarzhttps://redplanetlabs.com/learn-rama https://blog.redplanetlabs.com/2023/08/15/how-we-reduced-the-cost-of-building-twitter-at-twitter-scale-by-100x/ Stay in touch: ✉️ Subscribe to our newsletter: https://softwaremisadventures.substack.com 👋 Send feedback or say hi: softwaremisadventures@gmail.com Segments: [0:00] flying [0:07] inefficiencies of backend software development [0:17] suffering oriented programming [0:23] AI programming? [0:25] RAMA’s programming model [0:33] deployment & monitoring with RAMA [0:36] building a twitter clone at scale with RAMA [0:43] migrations with RAMA [0:54] driving adoption for RAMA [1:01] fundraising [1:15] building a fully remote team

Kelsey Hightower was a Distinguished Engineer at Google, where he worked on Google Cloud Platform. In this second part of the conversation, we focus on Kelsey’s retirement - the financial planning that enabled him to retire at 42, how he got started advising startups and his perspectives on compensation, turning down a substantial offer from Microsoft and meeting Satya Nadella in person. And, of course, plans for the future.

Kelsey Hightower was a Distinguished Engineer at Google, where he worked on Google Cloud Platform. In this first part of the conversation, we delve into pivotal moments in Kelsey’s career journey ranging from buying his first car by working at mcdonald’s after school, to starting his own computer store that turned into a music studio after 6pm, to hacking on python infrastructure with the core developers. Through these stories, we learned a ton about how Kelsey thinks about acquiring new skills - getting paid for it, breaking into the world of open source, navigating corporate politics, building trust within a team, and much more.

Julie Amundson is a Sr Staff Software Engineer at Google working on Machine Learning Infrastructure. Prior to Google, she was the Director of Machine Learning Infrastructure at Netflix. Julie decided to take a career break last year when she was affected by mass layoffs. In this conversation, we talk to her about what it was like to find a job during hiring freezes, what it was like to position herself in this market, whether the interviewers cared about the career break she took and how the career break changed her perspective towards work and life.

Chris Pruett is the CTO and Co-founder of Jam - a new way to share and listen to bite-sized audio. Prior to Jam, Chris spent 9+ years at LinkedIn growing from an engineering manager to VP of Engineering. During his tenure at LinkedIn, he worked on almost all aspects of the app and towards the end, led an org of 500+ engineers working on Feed, Messaging, Identity and Search. In this episode, we discuss how he made the decision to leave his leadership position at LinkedIn and co-found Jam. We also spoke about his time at LinkedIn and how he developed the practice to make value based decisions both in professional and personal life.