Alex Khurgin: I'd like to start with a few stats from Cindy Huggett who's been a great partner and advisor at home. She's done a lot of research into virtual training and puts out a report every year. Her most recent report for 2019 showed that although 93% of organizations are using some form of live online learning only 10% of programs are actually being administered live online. That means, until the beginning of this year, we had a big disconnect. If we were doing this workshop in January, we'd have had an interesting conversation about why there is a disconnect between all these folks using live online learning, but then COVID-19 changed everything. These changes might be permanent, at least for several years. There's been a lot of economic disruption, new demands on leadership, and the dramatic rise of remote work. Organizations were suddenly faced with the issue of providing training to their now dispersed teams. A year ago, those trainings would have been delivered in some other way, but now there’s been a massive ship to live online training.
A lot of you are encountering the challenges that come along with that. You've had to scramble to convert existing training programs to the new format of live online training. You’ve also probably encountered the four perennial high-level problems of workplace learning and training.
- It's not engaging enough. It’s difficult to make workplace training engaging especially when you compare it to the classes people choose for themselves to pursue their passions outside of the workplace. The engagement tends not to match up.
- It’s not effective enough. Workplace learning and training isn’t effective long-term. It doesn't lead to long term memory retention and, specifically, long term procedural knowledge. This means it often doesn’t lead to new habits.
- Even if it was engaging and effective, we're not measuring enough to tell.
- The remote challenge. Finally, all of these problems can be exacerbated when you're training remote teams.
Don't make these four mistakes. They basically boil down to not taking these four problems seriously enough, but can be avoided by investing in making training for your teams engaging, effective, and measurable. That said, all of you will have your own unique challenges and this interactive session is so we can actually talk about them. Before we move forward, we want you to reflect on those challenges so we're going to get into breakout rooms. In a minute, I'm going to place you in a room with two or three other participants and I'd like you all to take turns answering these questions:
- When it comes to delivering training to remote teams in 2021, what are your one or two top problems to solve or challenges to overcome?
- Why are they at the top?
- What would you need to shift for your organization to be able to address these challenges?
Let's break the seal here. Katie, what came up in your conversation?
Participant (Katie): Interestingly enough, everybody said their number one challenge was getting training to stick with participants after the program ends.
Alex Khurgin: We'll talk about that problem in the effectiveness section. What else? Any other insights or patterns that you started to observe? Rhonda, what came up in your conversation?
Participant (Rhonda): We talked about the challenge of follow-through because we're not there to observe any behavioral changes. The other issue was remote training taking a little bit longer than face-to-face. Because of that, we’ve had to try pulling out key learnings and priorities to reduce the amount of content we cover because there's no way to keep someone on a call for eight hours.
Alex Khurgin: All right. Let's address all of those things. You may think these things you’re facing are unique to your organization, but many businesses are experiencing the same challenges.
So, we went through the problems with live online training and talked about your personal challenges. Now, for the reason that we're here and the reason Hone has been doing live virtual training since about 2017. For the last three years, we've made a lot of different mistakes and struggled with engagement, effectiveness, and measurement, but we’ve also learned a lot. For the remainder of this session, I'm going to share the most valuable lessons we've learned and how they can help your organization. We’ve found that if if you do live online training the right way live, it’s actually can be better than classroom training in a lot of ways. It's more cost-effective, scalable, inclusive, flexible, safe, and ultimately, more effective. Here's how we can make that a reality.
We have this metaphor that we like to use called a learning cascade. Learning starts with some live experience, a live class that's supported by practice or a capstone project, or maybe some triggers to do the action. After that experience, people have access to just-in-time performance support, like checklists, cheat sheets, templates, and quick reference guides. If those resources don't work for somebody, then they can dive more deeply into self-paced learning and reinforcement. Then, they need more, they can re-enter a live class. It's a cycle that can happen again and again over time. Learning requires you to take that plunge. Why do you think it might be a good idea to start with a live class rather than just jumping into self-based learning?
Participant: To establish a connection?
Alex Khurgin: Yeah, absolutely. One thing that happens is you get an emotional connection to the content and to the goals of the training. What else might be a reason why we start with a live experience?
Participant: To build on the previous comment, to establish context.
Alex Khurgin: Yeah and that context can be pretty broad, it could be more context around the actual training itself or where can you get performance support materials, like where can I access the self-paced learning and reinforcement Absolutely, those are all good reasons.
The problem with starting with the just-in-time performance support or self-paced learning and reinforcement is that most people are poor judges of what learning methods work. It seems great, it's cost-effective and you can just put out a bunch of videos and content for people, but what ends up happening is people just half-listen to a video or half-read something and think that's enough. But, the video can’t answer your questions and it can’t check your understanding of a concept and there's only so much you can get from giving someone a multiple-choice question. All these things can be resolved if you have a virtual experience led by a seasoned facilitator who's cued into when people are paying attention and what they do or don’t understand. Like all of you mentioned, what you get is this extra context, motivation, and feedback that makes learners feel safe to make mistakes. All of that can happen in that live experience leading up to the rest of the resources.
There's a poem written by Alexander Pope that begins with these two lines, “Little learning is a dangerous thing; / Drink deep, or taste not the Pierian spring.” This is the idea that learning a little bit is actually more dangerous than not knowing anything because you think you know a lot and have this illusion of learning. Imagine watching one video and thinking, “Oh, yeah, I know how to give feedback now,” or “I know how to hire people.” How do we make sure that people are not drinking shallow, they're drinking deep? For the rest of this session, I'd like to focus on concrete tips for how to ensure your employees are engaged, retaining information, and taking action of lessons they learned from your live online learning experiences.
Lisa asked, “How does performance support differ from self-paced reinforcement?” An example of performance support is something like a quick reference guide or a checklist outlining the main steps of giving feedback again. It’s a resource students can quickly use. Self-paced learning and reinforcement might be watching a few videos, taking a quiz, reading some articles – going a little bit more deeply into a topic.
Before I reveal our top tips for engagement. Let's take a second to answer this question in the chat. What are some things that you've seen that could increase engagement with live virtual training? Let’s take about a minute.
Lisa mentioned activities. Absolutely. Someone else said activities, lots of interaction, constant interaction, seeing live faces —these are all great. We're going to talk about all of these breakout rooms, role plays, discussions. You all are experts and you're in the right place.
Here are the things that we've observed or that are the most important. I'm breaking these out by what you can do in the learning experience, what you can do as a facilitator, what you can do with the technology, and then what you can do as an L&D or HR leader, even if you're not actually designing any of the training. The first, most important part, which we've already kind of alluded to is what do you do? What do you do if you have eight hours of training? Well, unfortunately, you have to keep the session short, which largely means 60 minutes or less. When I first started at Hone, I wanted to work on 75- and 90-minute workshops, but after 60-minutes you really start to see diminishing returns. When you start to get into multiple hour-long training, you're better off just breaking sessions up over time. That's probably better anyway because breaking sessions up gives you more time to let the knowledge sink in and start to stick. It’s also more time for reflection and practice. Within that hour session, be sure to have constant interaction every three to five minutes. There should be something you ask the audience to do, like answer a poll, have a discussion in a breakout room or collaborate with another student. Introduce this interactivity early on and people start to get comfortable with it. Whenever possible, ask questions before presenting the material. And when you're presenting class material, do so in short, bite-sized steps.
I will admit in this session there's a lot of content, more than you would see in a typical workshop that we would provide. Usually, we break down sessions into three to four session sprints. So, if we had a whole sprint on “Designing effective learning experiences for remote teams,” we might have one session focused entirely on “Building engaging experiences.” The second session might be focused entirely on “Effectiveness and making learning stick.” The third session might be focused entirely on “Measurement,” and so forth. We found that creating these sprints gives people one central idea to work on at a time in our 60-minute sessions every week. That format has worked really well to keep people engaged and create a spacing effect that increases learning and helps things stick, rather than forcing everything into one day.
When it comes to facilitation, there are lots of small things that we found to make an enormous difference. One thing that you’ve probably noticed is that I’ve been calling everyone by their names, right? Nobody wants to participate, but as soon as you call someone by their name, they perk up. It's engaging and it's easier to do in a smaller session, right? Having a camera-on expectation is also part of that social engagement to make sure that people are paying attention. Set the ground rules for interaction upfront and make sure that people aren't multitasking. For example, these are the house agreements that we have at the beginning of every Hone session:
- Keep your video on.
- Be present; resist multi-tasking
- Active participation
- This is a judgment-free space
We have these house agreements so that everybody knows that to keep their video on and not to multi-task, right? We warn people they will probably be called on and if they do make a mistake, that’s no problem – this is a judgment-free space. When you set the ground rules, you want to make sure that everybody's participating. Usually, we'd have a session that's 10 to 15 people, so it's easier for the facilitator to keep track of who's spoken and who hasn't. Then, if you have breakout rooms, you can also make sure that everyone has a chance to practice. On Zoom, it’s possible for you to pop into different breakout sessions too, not to police people’s attention, but to ensure I could have if I wanted to pop into your breakout rooms, you know, not to police attention, but to ensure everyone's participating and having rich conversations, not penalizing their partners by not participating. Speaking of technology and breakout rooms, it's important to get to know all of the features that you have in your delivery tool. At Hone, we use Zoom, which has lots of rich, interactive elements that we can play with. If you see here on the left, this is also a chart from Cindy Huggett who has done great research on live virtual training. Zoom, WebEx, Adobe Connect, GoToWebinar – these are the most popular tools being used right now. You’ll have to research which one will work best for your organization. Again, this is in your learner guides, just in case you want to reference this slide later.
And then secondly, regardless of what tool that you're going to use, ensure everyone has a webcam and high-speed internet. Lagging and a poor connection can ruin the experience for everybody, especially if there's a lot of interaction built-in.
Finally, this is our last section on engagement and then we're going to have a discussion about this. There are many things that you can do to make a huge difference, even if you're not designing or delivering the training yourself. One that we've seen is that our customers that are active about getting buy-in prior to the program have been really successful. This might involve sending out an email or a form outlining the expectations of the program, whether there's going to be homework or a capstone project involved, and getting people to make a commitment to do that by signing on it. Likewise, we found it a lot easier for people to get buy-in when you're focused on what you're actually aiming for, what the behavioral changes are, what the cultural changes in your organization are, rather than the program itself. We're doing this management program so that we get to whatever it is, it'd be awesome if you can show up to some of the trainings as well, and encourage other leaders to occasionally drop in maybe at the beginning to show that people really care about this.
I've mentioned this a few times, but if you can focus on breaking up groups into about 10 to 15 folks, then the in-class collaboration is much more manageable. And you also get an opportunity to potentially update the program over time you get here's we have one wave of managers going through it this week. And then the next week, we have another way, but maybe in between each way if you can make program adjustments rather than giving everything to everybody at once. So let's discuss this. We have all of these different tips, right? It's a very large number and these are probably the most important ones, but there are other ones right of these. What stands out to you? Which one of these do you feel like you're going to invest in more? Let’s answer in the chat.
It might be something that you are already doing but could be doing more of, or it could be something you haven't been doing at all. Or mentioned short sessions, Kimberly mentioned checking for point of interaction calling people by name, and a lot of people are managing three to four session sprints.
Laura, what stood out to you about the three to four session sprints?
Participant (Laura): I think you do give people a break to really understand the content and to meet again. It's a continuation of learning. It just can’t be too long or people will lose interest.
Alex Khurgin: At first we were really ambitious and wanted to create these 12-session boot camps that people would take over six months. It was crazy, people would drop off after the fifth session. We found doing three to four sessions with a capstone project at the end is the sweet spot.
Lia asked how can you get buy-in prior to the program? How might you do that in your company?
Participant: I like the idea of just getting commitment to the ground rules as well. You need to make sure that they're both set up ahead of time so that when people enter they are prepared and you know that they've connected with it.
Alex Khurgin: Awesome. Thank you, everybody, for reflecting on this. Again, this is all in your learner guides and you'll also get a recording of the session afterward if you want to reference any of these.
Let's talk about the effectiveness of virtual training, which is the sticking point, right? No pun intended. The first thing we’ve learned when it comes to effectiveness is that we think about behaviors rather than topics. That opens up everything else to be more active and interactive. Behavior is it's not a one-off task or action. It's the way somebody acts in response to a stimulus. Great management behaviors include giving feedback, having coaching conversations, having one-on-ones with your direct reports, running interviews, leading performance reviews, etc. All of these things are behaviors of management and leadership regardless of what department you are in, you know. These are behaviors of a salesperson, product designer, marketing, etc.
Within those behaviors, it's important to focus on procedural rather than declarative knowledge. Can anyone tell me the difference between procedural and declarative knowledge or memory? Does anyone want to take a guess?
Participant (Lisa): Basically it’s the difference between knowing something intellectually versus being able to do it practically.
Alex Khurgin: Absolutely. That's right. Thank you, Lisa. Declarative knowledge refers to the fact that you know stuff that you can talk about. Procedural knowledge refers to things that you know how to do. For example, when it comes to bikes, something that declarative memory or knowledge might be knowing what the difference is between a mountain bike and a road bike.
On the left, that's what a lot of our training looks like. We have multiple choice questions that pertain to facts. On the right are procedural memories, or actually knowing how to ride the bike, which is probably more useful, ultimately. Declarative memories are easy to form, but they're also easily forgotten. As you probably experienced from your history class days, you don't remember too many of the specific facts. But procedural memories are more durable. We all probably remember how to ride a bike even if you haven't done it in a while. These memories are wired into our brains and our bodies. When we're trying to get people to memorize a series of management best practices or memorize how to give feedback, we're ignoring that there's a physical component to all these things, right? Who would want to receive feedback from a manager who knows to be clear and direct, but can't relay that criticism with compassion, or a salesperson who knows all the benefits of a product, but can't make you feel as if they have your best interests in mind, right? All of these interactions feel a certain way and if you don't do them in a certain way, they're not effective, even if you know all the background fat.
We need to focus on actions when we're trying to get stuff to stick and there are three really important elements that help us wire actions into people. First, it's having learners at the end of a session make a commitment and we'll see what that looks like. That’s another form of creating these expectations. At the end of the session, there's something concrete and actionable they can do.
Next, we've seen splitting learners into breakout groups, or what we call accountability groups, especially if you have a sprint of three or four classes with the same folks. They're going to be giving each other feedback, doing projects together, and practicing together. We have found also that splitting people up so they’re not always with your accountability group, is also a good way to introduce folks to others.
Finally, at the end of a sprint, we have a capstone project that has real-world applications. So let's take a look at this in action. And if you have any questions about this, please let me know.
Aiden asks for recommendations on how to maintain accountability with an organization on attendance adherence to ground rules. Yes, we'll get into that in the third section when we talk about measurement and accountability.
So here's an example from our How to Ask Powerful Questions workshop. Asking powerful questions is the behavior that we're targeting in this workshop. The first breakout room in the session is after the participants learn about Arthur Aaron's 36 questions that lead to love and they get to try them out in their breakout group. We just had a similar kind of breakout room where you all talked about your challenges when it came to training remote teams to get you thinking about and reflecting on the topic at hand, as well as get you interacting with and getting to know other people in your breakout group.
Later on in the session, there's guided practice. This is a guided case study where we have an argument between the sales team and the product team. We're learning about the kinds of questions we might ask to resolve the conflict and we do that together in guided practice. After folks learn about different question types and we talk about them as a group, then, they get to actually apply that by interviewing the facilitator. They get the information about the facilitator on the left, and then before they ask a question, they have to say, what kind of question is it. Like this, “Hey, Shiva, what's your favorite coaching topic? That's an example of an open question. Right?” They're practicing it again, in a kind of guided scaffolded way.
Then finally, there's a breakout room at the end where now that they've learned all these question techniques and looked at how questions work in the grow coaching model, they then ask questions from the grow model and follow-up questions in their breakout rooms. It comes full circle and you get this interactive experience all the way through. Some with your breakout room, some of it is guided and scaffolded with the facilitator, but there's always this interaction and wiring of procedural knowledge that we're practicing.
Finally, as I mentioned earlier, there's a commitment at the end. This, we actually link to a survey where they can fill out what their commitment is for the week. We get that information and sent back to them, saying, “Hey, this is what you committed to.” We've also seen folks opt into sending that commitment going to their manager saying, “Hey, your direct report has committed to this for this week, try to give them some feedback on it if you can,” to just make the manager aware.
At the end of that, there's a capstone challenge that's related to whatever class they’re taking. This is an example of the specific Capstone that was given for one of our clients. There are three things that everybody has to do and then they can choose what kinds of conversations they have.
Nice. Lisa mentioned, “I’m a big fan of the I do, we do, you do activity sequence.” Absolutely. It really is that kind of thing where you're modeling it for folks, then you're doing it together, and then you're letting them do it on their own with some performance support, right? Everybody's got the learner guide that they can reference if they need help. We're all going to practice this together in a second. If you haven't noticed, we're kind of modeling all these things at the same time.
The way to bake in those effectiveness boosters in the learning experience is to lean on your facilitators. For example, if you have a sprint, you can start a class with a review of the previous material. So session two, you start the class with a review of session one, you might ask people about commitments, right? “Hey, how's it going? You committed to this, right? How's your commitment going?” That's actually the default question we ask in every subsequent session after the first one.
The fourth session in a sprint is the capstone session where people just share their capstones, talk about how it went, and show off their projects depending on what they're learning. It might make sense to invest in some virtual facilitator trading because there are some folks that are really great in one context, they might be a really good manager, a really good virtual manager, or a really good in-person presenter, but they're not quite a great virtual facilitator. Maybe they don't know the tool or they don't have as much experience. So investing in some training might be a good idea there as well.
Lastly, before we actually go ahead and try this out on our own, some things that you can do to boost effectiveness as an L&D or a people leader are clarifying what the exact expectations are now that we've gone through the training. What are the KPIs, for example, if you're trying to get people to do one-on-ones in your organization? The KPI might be we want 80% of our people to report that they're doing consistent one-on-ones every week, by a certain date. That's a KPI that sets the expectation for everybody to focus on and direct their action toward. We also want to tie training to moments of a deed which we'll look at in one second. Just like you're getting people to commit beforehand and afterward, you're asking them questions throughout by simply asking, “Hey, how's this going?” You're encouraging application and you're providing feedback or you're encouraging your managers to provide critical and positive feedback to whoever is learning.
The moment of need is a nifty thing to talk about because it solves the problem with motivation. A moment of need is any moment that is getting people to feel like there's something urgent going on that they want to prepare for. I have an interview, final presentation, or performance review coming up and I've never done that, that's an example of a moment of need. Prior to that moment of need, people are super open to learning, they want to get ready for it, they want to brush up on it. During the moment of need, there's not that much time to learn, so they're just focused on performance support, but then afterward, they might reflect on how things went. “Could I have done that better? What problems can I fix?” But if you wait enough days after the event, odds are you don't care about anymore, right? There's this window of motivation that we want to focus on. That motivation is a really powerful way to get people to do things.
We want to track those moments of need across the organization and align our training to those moments. Those moments might be in your HRIS system, like when someone joins the organization you kick off onboarding. There might be recurring calendar moments, like annual performance reviews or quarterly planning that happens at the same time every year. There might be other recurring moments that happen randomly like an interviewer’s project kickoff, organizational changes, a project or product launch, or even global expansion. These are all high motivation moments where you can get people to care about learning, and to start changing their behavior.
And if there isn't a pressing moment of need, you can always create one from scratch. You can assign somebody a new project, give somebody a new responsibility, or declare a new company initiative to get people motivated. If we're focused on the skill of how to ask powerful questions, what would be a way for applying that new skill? Can you think of an example of that?
Participant: Could giving feedback be an example? Giving training on how to give feedback and then later on a manager has to deliver tough feedback to my direct report and says, “I need to brush up on this. Can you help me?”
Alex Khurgin: Yeah, absolutely. That's exactly right. Yeah. That's for the skill of giving feedback or the behavior of giving feedback. What about the behavior of asking questions? What would be an example of a moment of need you can create or call upon?
Let's get Marielle to answer. Are you here?
Participant (Marielle): I'm here.
Alex Khurgin: What do you think a moment of need might be for applying new questions skills?
Participant (Marielle): When someone is in need of learning a new process?
Alex Khurgin: Yeah, potentially, yeah. If they need to ask certain questions about that process to learn about it. Yeah. Another example might be if you have an interview coming up or if you have a performance review coming up, these are events where you're going to ask a lot of questions. Perfect. Now we're going to do the powerful questions training. If you want people to care more about it, you assign them something that has to do with it. Right.
So let's look back at the learning cascade because we're going to practice this together right now in breakout rooms. We have this live class, it's supported by all of this sticky stuff, practice, a Capstone, and triggers that lead to performance support then self-paced learning and reinforcement. If somebody needs more review, they reenter a live class. I'm going to put you back into breakout rooms and we're going to spend maybe seven minutes outlining a learning plan. You probably aren’t going to get through all of this, but challenge yourself as a learning professional.
*Pause for breakout sessions*
Alex Khurgin: Hopefully, this exercise was was fruitful for you even if your training doesn't exactly match this sprint framework.
Regardless of the kind of training that you're doing, you're going to need to do some measurement. The measurement model that we like the most is the Learning Transfer Evaluation Model. I'm very proud that it was developed by a friend of mine, Dr. Will Thalheimer, a couple of years ago. You may be familiar with the Kirkpatrick Four-Level measurement model, which actually, as it turns out through some digging by Will Thalheimer was stolen from Raymond Katzell in the 50s. There's this whole hubbub about it and where it actually originated, but no drama here.
We're going to focus on the Learning-Transfer Evaluation Model, which is more in-depth and has more tiers. As Aiden and a few others asked, “How are you tracking accountability at different steps?” Depending on the tier, you're actually tracking it in different ways. For example, at Hone we track attendance in an automated way, and this probably true for all of you as well depending on the tool you're using. We use our tool and session analytics that we’ve built to track activity based on how much people are talking. To understand learner perceptions, we look at the feedback we're getting. Some examples of that are all included in your learner guide, including a bunch of different questions that you can ask.
To evaluate knowledge, decision-making confidence, and task competence, we use the practice, capstone projects, and anything that has to do with knowledge transfer to measure success. When you're looking at the effects of transfer or did this actually work in our organization, we look at 180 assessments that we do before and after all the trainings.
I can't go through these right now, but I put these in here just for you to use. These are great questions that get at different things for so if you're focused on enjoyment, accessibility, or effectiveness you can ask these questions. Again, this is all on your learner guide.
I’d just like to thank everybody for joining us today. Before you leave, I wanted to let you know just a little bit more about Hone, especially the programs that we have to offer right now. If you're interested in converting your existing training program into live online training, that's something I'd be happy to help you out with. You can email me at [email protected]. If these meet your organization’s training needs or you'd like to consider a kind of sprint model that we've talked about, here are the classes we’re currently offering.
Again, please email me if you have any specific questions or if anything I mentioned was kind of relevant to your specific case, but you have some unique aspects you need help with, definitely email me. If you'd like any help with your management or leadership training, please go to tiny.cc/Honefeedback right now to fill out a quick evaluation form. I’d really appreciate it. It could also help crystallize some of your thinking about the session.
I'm going to stick around for another few minutes if anybody has any questions that you'd like to ask right now. Otherwise, thanks so much for joining everyone. Good luck with all your projects and thank you.
