Adventures In Evaluation Podcast

What does evaluation have to do with the movie The Big Lebowski? Well, the answer is "a lot" if you're up on this cult classic and number one favourite movie of James, aka "The Dude". Before you listen to this podcast we strongly suggest you view it first so you don't miss any of our keen insights. Cheers, Kylie & James P.S. Here is the photo of Kylie recording in the Starbucks bathroom while on family vacation.

This week we're thrilled to have Michael Scriven, frequently called "the father of evaluation", as our special guest. Michael chats about his chapter on "The Foundation and Future of Evaluation" in the new book, The Future of Evaluation in Society: A Tribute to Michael Scriven. Check it out here or subscribe to our podcast on iTunes for free. For more details on Michael's work you can visit his website here. Feel free to leave us your comments below, and don't forget to rate us on iTunes to give us some feedback! Enjoy, James & Kylie

In this episode we explore some of the differences between typical Western-style evaluation and the monitoring & evaluation (M&E) that occurs in an international development context. To help us, we invited two guests with experience working in both areas, Ian C. Davies and Tom Pellens. Here are some of the links we mention:John Mayne and Contribution AnalysisJennifer Greene and Mixed MethodsEuropean Evaluation Society 2014 conference Below is the cartoon mentioned by Terry Smutylo and Daniel Morales-Gomez. Until next time, Kylie & James

In this episode we present a mixed bag where Kylie debriefs her most excellent week at the African Evaluation Association 2014 conference and James takes us on his search for a "non-boring" evaluation journal article. Fun on the Conference Shuttle Bus Shout outs to Serge-Eric Yakeu Djiam, Jennifer Bisgard, and all the hard-working volunteers at AfrEA2014 and the Cameroon Development Evaluation Association for a great conference. Below are some of the articles that James found in his search: ‘What is Evaluation?’ Until next time, Kylie & James

Happy New Year Everyone!...okay...we're a little late with that...but we have a short new podcast! Many of us have personal New Year's resolutions, but what about professional ones? In this episode James and Kylie share their evaluation resolutions for 2014. Got some of your own? We'd love to hear them so leave us a comment below. Cheers Kylie & James P.S. Kylie just thought of another resolution she forgot to mention - use a systems lens!

In this podcast we chat with Kerry Bruce, Director of Results and Measurement at PACT, and an all-round expert on the use of mobile technology for data collection. Here are a couple of resources:resources in Mobile Data Collection Enjoy, Kylie & James

Have you joined the rubrics revolution? In this podcast we chat with Jane Davidson of Real Evaluation about the use of rubrics in evaluation. If this piques your interest, we have a slew of resources for you:Evaluation Rubrics: What, Why, and HowThe Value of Rubrics in Messy Non-Profit Evaluation ContextsEvaluation Rubrics: How to Ensure Clear and Transparent Assessment That Respects Diverse Lines of EvidencehereEvaluation specific methodologies: The methodologies that are specific to evaluation Unlearning Some of Our Social Scientist HabitsActionable Evaluation Basics: Getting Succinct Answers to the Most Important Questionshere Finally, if you've used rubrics in evaluation, please leave us a comment below and let us know how it worked for you. Until next time, James and Kylie

Couldn't make it to AEA2013? In this episode we share our own personal highlights of the conference. By the way:herehere Finally, don't forget to check out Chris Lysy's hilarious cartooning of the conference. Enjoy, James & Kylie

This week we talk to Sheila B. Robinson about the great AEA365 Blog! cheers, Kylie and James

Stephanie Evergreen's new book on data visualization came out yesterday, so we asked her to join us again on the podcast to give us some highlights plus her thoughts about the role and future of data visualization in evaluation. You can purchase a copy of the book here. See you at AEA! Kylie & James

Anyone in the mood for a horse race? Today we tackle the issue of different and competing evaluation approaches, models, types, whatever you want to call them, plus the challenges of inconsistent terminology in our field. Here are the two articles we mention by Paul Duignan: Terminology in evaluation: Approaches, types (purposes), methods, analysis techniques and designsUntangling Evaluation Terms – Discussing Evaluation 'Types' With Clients Often More Useful Than Evaluation 'Approaches' Here's the Evaluation Typology map that Kylie mentioned. If you're interested in the free Evaluation Glossary Mobile App you can download it here. As usual, we'd love to hear your comments below. Until next time, James and Kylie

This week we're thrilled to have Hallie Preskill join us to give a shortened version of her keynote speech at CES2013 in Toronto. Here are links to some of her current work, plus the Developmental Evaluation graphic she mentioned:Evaluating Social InnovationBreakthroughs in Shared MeasurementBuiding Evaluation CapacityReframing Evaluation Through Appreciative InquiryDevelopmental Evaluation graphic (full article) And here is this link to the book she mentioned on Big Data. Don't forget to leave us your comments below. James & Kylie

Are you a technical specialist, or a social change agent? Or as Dan Wilson at the Ontario Trillium Foundation says, "Are you part of the Rebel Alliance, or a Jedi Knight?" In this episode, we discuss the changing role of evaluators based on provocative keynotes from Hallie Preskill and John Gargani the CES2013 conference in Toronto. As promised, here are some of the links we discussed:Actionable Evaluation Basics: Getting succinct answers to the most important questionswebinarsbookSROI Canada John Gargani's Evaluator Doll John Gargani's slide on evaluators as social change agents. Jane Davidson discovers Tim Horton's. Anita Myers using data clicker at final plenary. Don't forget to tell us your likes and dislikes! James & Kylie

In this episode Kylie interviews James and Jane Whynot about their super cool initiative, Evaluation Mentoring Canada. Check out the website, and sign up: http://evaluationmentoringcanada.ca/. Cheers, Kylie & James Kylie and her first mentor, Dr. Anita Myers, at CES2013.

Hi Everyone, This week we've got Dawn Campbell-Borland and Dayna Albert as guests giving us a sneak preview into this year's CES conference in Toronto, June 9 - 12th. This year's theme is "Evaluation Across Boundaries". They have some great workshops, keynotes, and presentations lined up, so have a listen! For more information about the conference please visit http://cestoronto2013.ca/. Cheers, Kylie & James

In this episode we chat with Heather Buchanan, evaluation volunteer extraordinaire about the history, development, and use of the Canadian Evaluation Society's Competencies for Evaluation Practice. Here are some related links: Competencies for Evaluation PracticeCES Professional Designations ProgramEstablishing Essential Competencies for Program Evaluators Cheers, James & Kylie

This week we chat with Gail Barrington, Evaluation Consultant and author of Consulting Start up & Management: A Guide for Evaluators & Applied Researchers. Plus, here's a great little AEA365 post about small-sized consulting. Cheers, James & Kylie

It's March again, and evaluators everywhere are frantically racing to meet multiple deadlines. (In fact, Kylie almost forgot to post this episode herself.) Have a listen to James and Kylie discussing some of their techniques for dealing with March madness. How do you deal with it? Leave us your comment below.

It’s March again, and evaluators everywhere are frantically racing to meet multiple deadlines. (In fact, Kylie almost forgot to post this episode herself.) Have a listen to James and Kylie discussing some of their techniques for dealing with March madness. How do you deal with it? Leave us your comment below.

Some of us evaluators find ourselves using our stats straight away out of school, and for various reasons some of us don't. In this episode we discuss the different levels of use among evaluators and what is really acceptable in terms of statistical knowledge. Here's the link to Paul D. Ellis' great little e-book, Effect Size Matters. AEA often has eStudy courses on statistics, and here are some other links for online refreshers: Online Statistics Education: An Interactive Multimedia Course of StudyIntroduction to Biostatistics 1University's Statistics 1Introduction to Statistics This list is by no means comprehensive, so please, if you know of others, suggest them in the Comments below! And yes, it is Rip Van Winkle.