Packing up the spring 2020 semester and transitioning to
remote teaching was difficult, not just for me but also for my students. As I
told them, “None of us signed up for online teaching. But we’ll make it through
this.” And we did! At least most of us did…

Part of the challenge was doing this in my first year at
UW-Stout. I was prepping two courses (undergraduate stats and graduate
evaluation), and practically prepping my two sections of intro psychology after
a major overhaul from the fall. I was trying to keep up my scholarship and
service commitments. I was already a little burned out from the semester and
had to skip my spring break to focus on transitioning to remote teaching. I was
trying to figure out how to work from home again after finally getting used to
working in my department every day.

On reflection, I think there were some things I did really
well with the transition that I want to keep in mind for future semesters.
There were also some things I struggled with and want to improve for next time.
Here are the five things I am taking away from my experiences with remote teaching
due to Covid-19:

1. 
Have more lenient late policies

I already had a fairly lenient late policy: the points
possible decrease every day an assignment is late, then it’s maximum 50% credit
but you can turn it in at any time during the semester.

After the transition, I made it even more lenient: turn in
anything for full credit by the last day of the semester. In other words, there
was no late policy. Deadlines were all suggestions.

Students were grateful for both structure of assignment
deadlines and for leniency when they couldn’t meet them. Some students needed
the regular deadlines, whereas others needed to focus on other courses before
they could think about mine. And you know what? It worked out just fine.

I was always a little worried about abuses to the policy. Would
students get the answers from another student to submit their homework? Would I
get a huge influx of grading right at the end of the semester? Neither seemed
to happen.

This policy doesn’t work for all situations, particularly
when a large project is broken up into multiple sub-projects. But I plan on
being much more lenient in the future.

Relatedly, I’m going to think more critically about
high-stakes assignments. All my classes went to low-stakes assignments
throughout the semester, and it was much more enjoyable for both students and
myself.

2. 
Incorporate more videos and flip the classroom

For my undergraduate stats class, I recorded all my attendance-optional
lectures and put them into our LMS. This didn’t require any additional effort
or time beyond adding the link to the LMS after the recording was done.

Yet it saved me so much time answering student questions. Students
learned pretty quickly that most of their questions could be answered by
referring back to the lecture, so I wasn’t fielding as many repetitive
questions about the basics. Instead, I could focus on the more advanced
questions students were asking about the content we were learning. It was so
much more rewarding!

I will definitely be incorporating more videos into my
classes in upcoming semesters, regardless of whether we’re online or in person.
I want to design my courses to be disaster-ready: flipped so that much of the
learning is on their own and class time is spent applying the content. That
way, if something does happen, they’re already set to finish out the semester
the way they started the semester.

p.s. If you’re looking to improve your videos, I finished
reading Karen Costa’s “99
Tips for Creating Simple and Sustainable Education Videos” and I highly
recommend it.

3. 
Build a better classroom community

I was particularly proud of the community that I build in my
intro psych classes using team-based learning. But Covid-19 hit just when teams
were starting to norm and perform. To accommodate students’, I went completely
asynchronous and made the class as easy as possible for them to complete (and
still I had a number who sadly were not able to complete).

But the one thing I struggled with was keeping up the
classroom community we built when we were in-person. This was a struggle for
all my classes, even my graduate class that kept up synchronous meetings. I
just felt like I wasn’t connecting with a lot of my students anymore. I couldn’t
check in with them before, during, or after class like I was doing prior. And
email check-ins just weren’t the same…

I’m still not entirely sure how to go about this. I have
some ideas though: more videos to humanize myself and connect with students,
especially if we’re online; having an assignment for points that has students
come visit me during student hours, whether that be in my office or online;
continue to use MS Teams for each of my classes and grading on participation; and
continue to hold online student hours, even if I have in-person student hours. I’d
love other suggestions you have for building a community in an online
environment.

4. 
Embrace universal design for learning

The transition to remote teaching made me better embrace UDL in my classes: providing multiple
means of engagement, representation, and action and expression. I was already
doing it a little bit, but I realized how important it was for my students that
they have alternative ways to access and participate in the learning opportunities.
  

For example, I had two small papers in my intro psych class.
At the beginning of the semester, I decided to open it up so that students
could either write an essay or they could record a video. After the transition,
I had a student ask to just call me and describe what they did, which of course
I allowed.

I still have a lot to learn about UDL—and the CAST website provides a ton of great guidelines and resources—but one of my core principles in teaching is that my teaching is accessible for all students.

What about you? What lessons are you taking away from the transition to remote teaching due to Covid-19? What changes will you make to your land-based teaching moving forward? Add your comments below!

In reply to Allison Titcomb.

Certainly, thinking of evaluation as a transdiscipline or the alpha discipline that Scriven describes is one reason why I think evaluation can inform research-practice partnerships. I think it’s also just the consultancy nature of what we do and the importance we place on interpersonal factors. like communication and relationship building.

Notion calls
itself an all-in-one workspace for notes and documents, wikis, tasks and
projects, and spreadsheets and databases. Although I still prefer Todoist as my
task manager (I wish Todoist and Notion could integrate!), Notion has become my
project management software tool.

I’ll admit: the learning curve on Notion was a bit rough at
first. It took a while to understand how the databases work, how pages fit in
with databases, navigating the workspace, and collaborating with others. I
doubt I am using Notion to its full potential right now. But right now, it’s
working perfectly fine for what I want it to do.

Below, I will detail four ways I am using Notion: tracking publications, my thesis students’ progress, summer goals, and notes about the courses I teach. (Click on the links to go straight to that section.)

Notion for tracking publications

This is the simplest way I am using Notion is through tracking my publication pipeline. I found a template on Notion and adapted it to my needs. After reading a recent blog post by Chelsea Hetherington, I added tracking the journal, publication date, PDF, and journal URL. I imagine this will be useful for sharing publications when people ask me for the PDF or location of an article. Use this template to track your publication timeline by clicking the “duplicate” button on the top right of the page here.

Below are the two views I currently use for my publication timeline. The first is the table view, in which I can easily see my role, status, and publication information.

The second is the timeline view, in which I see it based on what status the publication is in. Note that under “Properties” I could choose to add showing my role of the publication if I so choose.

Notion for tracking theses

I adapted the same timeline template above for tracking
theses across the main stages of the thesis (ideas, proposal, data collection, data
analysis/reporting, finalizing). Names are crossed out for privacy.

Each thesis student has their own page so I can track them
along the timeline. On this page, we can add comments to each other, add a wiki
or notes, link to other pages, and so much more.

The bottom link is the most important. This links to another
page that is a more detailed tracker of their thesis. I have a template based
on the general timeline to get students finished by graduation within the two
year program. I let the student track and add things to this timeline and keep
us updated. For example, this student ended up creating a table to organize
their thesis literature, which was added to the template. Use
the template for this thesis tracker by clicking the “duplicate” button on the
top right of the page.

Notion for tracking summer goals

After taking a workshop by Cathy Mazak on staying on top of
summer writing for academics, I sat down with my list of projects for the
summer, developed all the tasks needed to accomplish those goals, and created a
rough timeline of when each task should get done. Use
the template for this goals tracker by clicking the “duplicate” button on the
top right of the page.

I have many views I use for this tracker. I like to View All
when I’m adding new tasks.

I like to view by Status to hone in on what I am currently
working on this week. I plan this out every Sunday to ensure I’m always moving
projects forward.

I like to view by Due Date to get a general sense of what I’ve
accomplished each month and make sure no month is too heavy.

Lastly, I like to also look by Project to see the list of
tasks for each project. Notice I can change the properties shown as well as the
sort and filter. If I only want to see tasks that aren’t “Done” or for a single
project, I can use the Filter feature. If I want to sort by due date or task
status, I can use the Sort feature.

Lastly, there is a calendar view that I could use, but
I haven’t found that useful because I prefer to add these tasks to my to-do
list on Todoist and track the individual tasks there instead. So I usually only
look at this once a week.

Notion for tracking course notes

Lastly, I have been using Notion to keep notes on all the
revisions and notes I make about my courses throughout the semester and as I’m
prepping. I had not yet found a satisfactory way to track these notes and
thoughts until now! Whenever I have a thought, I can just post it here, add
which class(es) the note refers to, add a URL if necessary, and even add more
comments and notes by converting the note to a page (the third and fourth items
have a page icon on the left, which indicates there are comments inside as a
page). I can then check off if I’ve reviewed the content already for when I’m
doing course revisions. As you can see, I’ve already reviewed all the material
for PSYC 570 which I’m teaching over the summer (a course on interpersonal
effectiveness which I’m incredibly excited about!). Use
the template for this course notes tracker by clicking the “duplicate” button
on the top right of the page.

If you find these templates useful or have questions, let me know in the comments below!

The RoE TIG convened a working group to help promote the
field of RoE, and one of our first tasks was to come to a consensus on the
definition of RoE.

There are many definitions of research on evaluation (RoE), which
can make it difficult to understand. Through searching the literature and
discussing with colleagues, we found four definitions (presented chronologically):

1. [Any] … systematic inquiry into the methods,
   practices, and profession of program evaluation, with potential implications of
   its findings for evaluation theory (Brandon & Fukunaga, 2013).
2. Systematic empirical inquiry resulting in
   original findings or reexaminations of existing data about the practice,
   methods, or profession of program evaluation (Brandon, 2015).
3. Any purposeful, systematic, empirical inquiry
   intended to test knowledge, contribute to existing knowledge, or generate new
   knowledge related to some aspect of evaluation processes or products, or
   evaluation theories, methods, or practices (Coryn et al., 2015).
4. A research investigation that generates findings
   with the intended purpose of creating a stronger evidence base and
   infrastructure for the applied practice of evaluation (Fierro, working
   definition).

The first three definitions have a lot in common: they are
focused on systematic inquiry using empirical methods aimed at
examining a variety of aspects of evaluation, including the methods,
practices and processes, products, theories, and profession.

The Brandon and Fukunaga (2013) definition also discusses
implications for evaluation theory and the Coryn et al. (2015) definition
discusses implications of testing knowledge, contributing to existing
knowledge, or generating new knowledge. Essentially, these
definitions point to RoE as research for research sake.

But the Fierro working definition suggests that RoE is for
more than just creating a stronger evidence base, one that seems built by and
for researchers. Rather, RoE is also for building an “infrastructure for
the applied practice of evaluation” (emphasis added).

It is for that reason we adopted the Coryn et al. (2015)
definition in conjunction with the Fierro working definition. The definition
might look something like this:

Research on Evaluation (RoE) is any
purposeful, systematic, empirical inquiry intended to create a stronger
evidence base and infrastructure for the applied practice of evaluation.

We believe this broader definition both (a) encompasses
everything the Brandon and Fukunaga (2013), Brandon (2015), and Coryn et al.
(2015) definitions propose and (b) emphasize the importance of RoE for
researchers and evaluators alike. RoE is not conducted for the sake of conducting
it, nor is an evidence base of research important unless it is useful
and used by the intended audience—in this case, practicing evaluators.

We hope this definition encourages researchers of evaluation to pursue topics that not only create a stronger evidence base but also informs evaluation practice.

Special thanks to everyone in the working group who has been involved in this so far: Kathleen Doll, Michael Harnar, Nina Potter, Eric Barela, Gregory Greenman, Esther Nolton, Ramjee, Miriam Jacobson, George Harrison, Seema Majato, Leslie Fierro, and more!

Although there is much to change in my intro to psychology course I taught for the first time this semester, one thing I know I will keep next semester is the concept demonstration project that students did at the end of the semester. For this project, students were told: Your main objective in the concept …