Katelyn Talbott, University of Illinois Urbana-Champaign

Karin Readel, University of Illinois Urbana-Champaign

Research indicates that graduate students are best served when they have frequent interaction and communication with advisors and faculty about their academic progress (McCuen et al., 2009; Schroeder & Terras, 2015). Providing graduate and professional students with annual feedback via progress reports is an integral component of student success through clarifying expectations and providing an opportunity for early intervention in areas of concern. Though most graduate programs require annual progress reports, there is variation in the information requested of students and how that information is used to improve both student and program outcomes.

Annual Review Framework

When conducting an annual review, the data collected must be meaningful and fulfill the needs of various stakeholders including faculty advisors, academic advisors, review boards, program administrators, departments, colleges, and more. On the flip side, from a student’s perspective, the annual review should be easy to complete, structured, and have a clear outcome as to why they need to complete the annual review. 

The authors have developed an annual review framework based on their experiences and challenges in their current roles working with masters and doctoral students. Outlined below is the process of an annual review, followed by multidirectional flows of the annual review—an administrative flow and a student flow, respectively. These three pieces make up what the authors have coined as the annual review framework. This framework was developed and is in use at a Research 1 institution in the Midwest. This process is followed by two interdisciplinary PhD programs (170+ students) and a professional masters program (600+ students). Both programs require the completion of an annual review by both the student and (faculty) advisor. 

Figure 1

Annual Review Framework

The Process of an Annual Review

1. Instructions
2. Annual review
3. Results

This annual review framework follows the annual review process using two flows. One flow, following the processing in reverse chronological order, is the administrative flow. The second framework, student flow, follows the annual review process in chronological order. 

Administrative Flow

The administrative flow of the annual review framework is how the annual review is created, updated, communicated, and maintained. With a functioning administrative flow, all stakeholders will be able to have their demands met effectively and efficiently. Remember, the administrative flow follows the annual review process in reverse order. This allows for large-picture needs and timelines to be taken into consideration and actively implemented in the annual review. 

3. Results
Stakeholder needs: To ensure the annual review collects meaningful data and the students do not experience survey fatigue, the needs of the stakeholders need to be considered while developing the annual review. Stakeholders include students, faculty advisors, academic advisors, and program administrators.

Factors to consider: Faculty needs for grant/lab funding; student academic status for financial awards, scholarships, or fellowships; international student status; review board evaluations; programmatic reports including assessment; published demographics; accreditation purposes. 

2. Annual Review
Instrument development: This is the stage in which the annual review is physically developed for students to complete. Attention should be given to the accessibility of the annual review. The timeline also needs to include an ample testing period that involves actual end-users (students, faculty and staff).

Factors to consider: Delivery platform; length; paper or electronic annual review; external links; will documents like a resume or CV need to be uploaded; preferred format or maximum size document that will be accepted; ability to resubmit or edit initial responses; how final results will be archived or reported to stakeholders.

1. Instructions
Logistics of review process: Now that the information that needs to be collected is set to successfully be recorded, attention needs to be given to the actual process students will follow to complete the annual review. 

Factors to consider: A website or webpage with instructions and links to download or electronically complete the annual review; templates for students to use for any free-response questions; faculty template letters indicating a student’s satisfactory/unsatisfactory degree progress; timelines clearly communicated to all stakeholders for the annual review period and any additional steps or meetings that are required to complete the process in full; built-in reminders to both faculty and students.

By following this administrative flow, an annual review process will be well developed to meet the needs of the review. Such a thought-out process has led the authors to increased accountability from the faculty and students as well as a process that is time efficient. With an established administrative flow, the second part of the process needs to be just as well established and thought out: the student flow.

Student Flow

1. Instructions
Communication and student preparation: Once the annual review is ready to be deployed, students need to be notified regarding what it is, its importance, the expected timeline for its completion, and what they need to do in preparation. Communication about the annual review process should take place in multiple venues, such as the program handbook, routine emails, blogs, wikis, newsletters, orientation sessions, etc. Reminders must be built-in to ensure compliance with deadlines and students need to be informed of the potential consequences of failing to complete the process.

Factors to consider: Where to publish information about review process; timeline constraints; delivery and timing of reminders; impact and consequences of failure to complete form. 

2. Annual Review
Self-Evaluation and Feedback Meeting: In this phase, students actively enter the requested information and then schedule/hold a meeting with their advisor to review their form. It’s important to time this process correctly to avoid competing demands on faculty time, semester breaks, commonly attended annual conferences, etc.

Factors to consider: Provide templates for open-ended questions; provide sample CV format to aid in data collection such as publications; timeline for completion (for both student and faculty); standard questions to guide student/advisor meeting; build in process for students to revise form.  

3. Results
Letter from advisor: The outcome of the annual review process is a letter from the advisor indicating a student’s satisfactory/unsatisfactory progress toward the degree. Letters are reviewed by program administrators to ensure guidelines are followed before being placed in student files. 

Factors to consider: Impact of the letter on students’ academic standing (probation? dismissal?) or funding; logistics of probation decisions in the graduate college; the impact of letter on advising relationship; future progress plans for academic work and milestone exams; who reviews letters and distributes to student; plans for following up on unsatisfactory letters.

The authors now challenge you to think about how the annual review process, with its administrative flow and student flow, can be adopted at your program and institution. Setting up such a process could be administratively intensive, but once it is set up maintenance is less cumbersome than re-creating the process each year. 

Recommendations for Successful Implementation

In the process of developing the most fitting annual review for a program, administrators need to be wary to not inundate students with numerous surveys, request students complete surveys for the sake of checking the box, or collect data that is not reviewed, analyzed, or acted upon promptly. To counteract these faux pas, the presenters encourage you review these recommendations for successful development and implementation of an annual review by students and faculty/academic advisors:

1. Fitting the annual review into existing requirements such as a required advising meeting, registration time, or yearly goal-setting meetings.
2. Ask only questions that are relevant to the data you wish to know. If you only need to know what conferences a student presented at, ask that upfront. Don’t also ask where the conference was, how long they were gone, and request that they submit any presentation material.
3. Coordinate timelines and procedures with pertinent offices on your campus. For example, annual reviews need to be completed with sufficient time to meet university probation or program assessment deadlines.
4. When developing the process, build in survey analysis time. With a set timeline for annual review development and student completion, ensure there is time provided to review the results—especially if the review is tied to academic standing or has financial implications.

Conclusion

The annual review framework presented in this paper outlines a process for incorporating the needs of multiple stakeholders in the development of an annual review for graduate students. Careful consideration of both the administrative and student needs will result in an instrument that not only provides necessary information, but also increases the student experience by helping them feel more connected and engaged in the program overall. This framework also allows for a strong advisor presence, since advisors are included in programmatic goal setting and data acquisition phases initially, and not as an afterthought.

References

McCuen, R. H., Akar, G., Gifford, I. A., & Srikantaiah, D. (2009). Recommendations for improving graduate adviser-advisee communication. Journal of Professional Issues in Engineering Education and Practice, 134(4), 153–160. https://doi.org/10.1061/(ASCE)1052-3928(2009)135:4(153)

Schroeder, S. & Terras, K. (2015). Advising experiences and needs of online, cohort, and classroom adult graduate learners, NACADA Journal, 35(1), 42–55. https://doi.org/10.12930/NACADA-13-044