Resources for Needs Assessment
Get
the Horse Before the Cart: Conducting Assessment of Advisor Development
Needs
Authored by:Terry Musser, Tracy Hoover and Marcos Fernandez
2008
A needs assessment is a systematic way of determining the current state of an organization before developing solutions or programming. Needs assessment conducted to obtain baseline data on the service needs of a particular population can save an organization money and time. When organizations decide to save time by skipping this important task, money and human resources can be wasted on implementing solutions that did not hit the targeted population where it could be of the most benefit. A needs assessment can be proactive or reactive. For example, if yield or retention numbers are decreasing, a needs assessment could be used to determine the cause or possible solutions. On the other hand, a needs assessment can be conducted before there are any negative indicators or identified problems simply to identify future programming needs.
A comprehensive advising needs assessment can provide valuable information such as:
- advisor perceptions of what advising means
- advising techniques and resources currently being used effectively
- advising techniques and resources NOT being used effectively
- identification of the professional development needs of the advising population
- identification of additional resources needed by advisors
There are well-defined steps to take to design an effective needs assessment.
- Determine the purpose of the needs assessment. What are currently available for conducting this assessment?
- Identify the necessary resources. What resources are currently available for conducting this assessment? Is there assistance with survey development and implementation at your institution? Do you have/need help with the distribution, collection, and analysis of the assessment?
- Decide who will be responsible for designing, implementing and analyzing the assessment.
- Determine what research questions must be included in the assessment.
- Decide on the methodologies for distributing, collecting and analyzing the assessment.
- Establish a workable timeline for accomplishing each task.
There are many data gathering methodologies available and appropriate for administering a needs assessment. In most cases, the research questions and the purpose of the needs assessment will drive the identification of methods that will be most effective. For example, if the goal is to ask one or two questions and allow group discussion to steer the final results, then a focus group interview could work. If a standard set of questions to ask of a specific population has been identified, then a survey might work best. Individual interviews, or, on the opposite end of that spectrum, town-hall-meetings, could be useful depending on your purpose and the resources you have available. One important step of any needs assessment is to determine what data already exist so you aren’t reinventing the wheel and are pushing beyond what is already known. Therefore, a mining of existing data is always crucial in terms of best resource management.
Of course, one always needs to keep in mind the final outcome, the final report, when planning a needs assessment. Who is the audience for the final report and how can that audience be persuaded of the importance of the needs assessment results? What do you want to have happen as a result of this assessment? This final step must be carefully planned to achieve the desired results.
A Case Study: An Advising Needs Assessment(Reprinted with permission from The Pennsylvania State University College of Agricultural Sciences)
Context
The College of Agricultural Sciences (CAS) at Penn State University uses a faculty advising model as its primary advising delivery mechanism. Several departments within CAS also employ a professional advisor to work with students and with faculty advisors within the department, though not all departments have this resource. One department employs a staff assistant to conduct undergraduate advising activities. As the advising coordinator for theCollege of Agricultural Sciences, the author is responsible for providing updates and training for all advisors in the College. The Associate Dean for Undergraduate Education, the Advising Coordinator, and the Department Head for Agricultural and Extension Education worked together to develop and implement a needs assessment in August of 2008.
This needs assessment was not conducted for the purpose of determining what might be causing a problem or because there were complaints about advising in the College. This needs assessment was conducted solely to help determine advisor professional development programs that would hit their mark in reaching the needs of the population. Recruitment and retention figures already told the story of positive results for the College. The undergraduate student population for the College is up 8% over 2007 and the average retention rate at PennState is well above 80%. This needs assessment was considered to be more proactive than reactive in nature.
Penn State has developed an online advising system called “eLion” that offers students the ability to schedule their own courses, run degree audits for any major, check their grades or tuition bills, predict their grade-point-average and many other academic tools. This system also has a unique set of tools for academic advisors, including electronic advising notes. Other online advising resources include the University Bulletin or catalog, a comprehensive Undergraduate Advising Handbook and advising handbooks specific to each college and/or department. The College of Agricultural Sciences (CAS) Advising Handbook includes semester-by-semester course scheduling recommendations or “Recommended Academic Plans”.
Methodology
The purpose of this needs assessment was to determine the advisor training needs of the 124 faculty and professional advisors within the College of Agricultural Sciences who have at least one undergraduate student assigned to their advising roster. Although advising workshops had been offered within departments and for the entire College in the past, there was no formal advisor training model in place and training was spotty at best. The needs assessment team discussed strategies and determined that an online survey instrument was the easiest, quickest and most effective method for gathering the needed data. Faculty in the Department of Agricultural and Extension Education had achieved good results conducting surveys using Survey Monkey (www.surveymonkey.com), an online commercial survey product that is easy to use and inexpensive. The team designed the online survey using the free trial version of Survey Monkey which limits the number of questions that can be asked to ten and the various types of tests and analyses that can be conducted. It was determined that these restrictions would not compromise the goals of the survey. The final instrument, then, was a simple ten-item survey that required minimal time on the part of the participant. The team went through an expedited human subjects approval process for publishing purposes through the University’s Office for Research Protections.
In mid-August, the instrument was piloted with four professional advisors in the College. Three of the four responded and made suggestions for improvement and clarity of the survey. OnAugust 22, 2008, an email invitation was sent to all 124 academic advisors in the College to participate in the study. Consent information was delivered through the email and the survey url was provided. No deadline for completing the survey was given, however a reminder email was sent to all 124 advisors again on September 5 and the survey was closed on September 22. A total of 54 responded to the initial or reminder invitation to participate for a 43.5% response rate.
Results
Ten questions were asked via the anonymous online survey: three demographic questions, four perception/attitude/likert-scale questions, and three open-ended questions. The results for each of the demographic and perception questions are presented below.
1. Number of years you have been advising at Penn State
|
Years Advising
|
<1
|
1-2
|
3-5
|
6-10
|
11-15
|
More than 15
|
Response Count
|
|
Responses
|
5.6%
(3)
|
13.0%
(7)
|
33.3% (18)
|
18.5% (10)
|
7.4%
(4)
|
22.2% (12)
|
54
|
2. Number of students you presently advise.
|
Number
|
1-10
|
11-20
|
21-30
|
31-40
|
More than 40
|
Response Count
|
|
Responses
|
24.5% (13)
|
43.4% (23)
|
13.2%
(7)
|
3.8%
(2)
|
15.1%
(8)
|
53
|
3. I am a:
|
Type of Advisor
|
Response Count
|
Percent
|
|
Faculty Advisor
|
45
|
83.3
|
|
Professional Advisor
|
6
|
11.1
|
|
Other
|
3
|
5.6
|
Other: Staff but not professional advisor (1)
Recruiter who volunteers to advise (1)
Staff that volunteers to advise (1)
4. Which one of the following best characterizes your attitude toward advising?
|
Advisor Attitude
|
Response Count
|
Percent
|
|
I find advising pleasant and rewarding.
|
43
|
79.6
|
|
I have neither very positive nor very negative feelings toward advising.
|
9
|
16.7
|
|
I find advising unpleasant.
|
2
|
3.7
|
When asked the reasons for their attitude toward advising, most respondents commented about the personal rewards they gained from developing relationships with their students. Those who offered reasons for negative feelings about advising highlighted that the few students who came unprepared and don't follow through with suggestions from the advisor can make the job frustrating.
5. Which one of the following best captures your perception of student attitudes toward the advising process?
|
Student Attitude
|
Response Count
|
Percent
|
|
Students find the advising process pleasant and rewarding.
|
30
|
56.6
|
|
Students have neither very positive nor very negative feelings toward the advising process.
|
20
|
37.7
|
|
Students find the advising process unpleasant and frustrating.
|
3
|
5.7
|
When asked to justify their perceptions of student attitudes toward advising, many respondents said that their advisees who take advantage of the advising process probably receive a lot of benefit from the relationship with their advisor. However, because advising is not mandatory at Penn Sate, there are those students who can either appropriately self-advise or who simply don't attend to their academic needs and never meet with an advisor. Several respondents commented that they heard students make negative statements about their advisor they had encountered.
6. What advising activities do you engage in the most (Check all that apply)?
|
Advising activity
|
Response Count
|
Percent
|
|
Course selection
|
51
|
94.4
|
|
Mentoring
|
40
|
74.1
|
|
Career/profession/graduate school considerations
|
41
|
75.9
|
|
Decision making, including choice of major
|
33
|
61.1
|
|
Academic procedures (i.e. late drop, withdrawal, Faculty Senate petitions, change of major)
|
41
|
75.9
|
|
Other
|
10
|
18.5
|
When asked to expand on the "other" category of advising activities, several respondents included working with prospective and transferring students as well as students enrolled at other Penn State campuses. The advising activity for these students would entail discussions about what courses to take before coming to the University Park campus and evaluations of student transcripts. Several respondents also identified personal or 'life' issues as a common advising activity.
7. What is your comfort level when using the following advising resources? Check the box that best describes your comfort level.
|
|
Very Comfortable
|
Somewhat Comfortable
|
Neither Comfortable nor Uncomfortable
|
Somewhat Uncomfortable
|
Very Uncomfortable
|
Have Never Used This Resource
|
|
eLion
|
64.8% (35)
|
27.8% (15)
|
3.7% (2)
|
3.7% (2)
|
0.0% (0)
|
0.0% (0)
|
|
Online Degree
Programs
Bulletin
|
38.9% (21)
|
24.1% (13)
|
14.8% (8)
|
7.4% (4)
|
0.0% (0)
|
14.8% (8)
|
|
Degree Audit
|
69.8% (37)
|
20.8% (11)
|
3.8% (2)
|
3.8% (2)
|
1.9% (1)
|
0.0% (0)
|
|
Electronic
Advising Notes
|
11.1% (6)
|
20.4% (11)
|
16.7% (9)
|
5.6% (3)
|
3.7% (2)
|
42.6% (23)
|
|
Schedule of
Courses
|
72.2% (39)
|
20.4% (11)
|
5.6% (3)
|
1.9% (1)
|
0.0% (0)
|
0.0% (0)
|
|
eLion
Transcripts
|
44.2% (24)
|
25.9% (14)
|
7.4% (4)
|
3.7% (2)
|
1.9% (1)
|
16.7% (9)
|
|
CAS Advising
Handbook
|
14.8% (8)
|
14.8% (8)
|
35.2% (19)
|
1.9% (1)
|
0.0% (0)
|
35.2% (19)
|
|
University
Undergraduate
Advising
Handbook
|
13.0% (7)
|
16.7% (9)
|
33.3% (18)
|
5.6% (3)
|
0.0% (0)
|
31.5% (17)
|
|
Department
Student
Handbook
|
50.0% (27)
|
27.8% (15)
|
11.1% (6)
|
0.0% (0)
|
0.0% (0)
|
11.1% (6)
|
|
Recommended Academic
Plans for Major
|
59.3% (32)
|
27.8% (15)
|
5.6% (3)
|
0.0% (0)
|
0.0% (0)
|
7.4% (4)
|
The last three (3) questions were open-ended. The first of these questions asked participants what they find most rewarding about the advising experience. Fifty-one (51) responses were received and virtually all respondents indicated that the most rewarding aspect of advising was helping students and seeing them grow, make progress and succeed. When asked what they find to be most frustrating or dissatisfying, twenty-six or 51% of the fifty-one respondents indicated that students who either don't take advantage of their advisor's expertise their frustration with the complex procedures involved and the lack of accurate information or knowledge about these processes. Six (12%) mentioned poor student attitudes as an issue, such as thinking that advisors should give them the answers and not make them think or that the advisor should respond to their questions immediately.
The last open-ended question asked advisors what types of personal or institutional support would make the advising process more effective or satisfying. Forty-one (41) responded to this last question with the following results:
More training (9)
Nothing is needed (7)
Rewards for advising (5)
Better departmental advising system (more professional advisors, more faculty engaged in advising, advising mentors within the department) (5)
Better online advising tools (degree audit, one location, streamlining of processes) (5)
Train the students about their role in advising (5)
Don't know (2)
Data on what techniques work with students (1)
A question center for advisors to get help (online) (1)
Discussion
So what did we learn by conducting this needs assessment? From the demographic information, we learned that more than 50% of our advisors have been engaged in advising for five (5) or fewer years and that the majority of the advisors have between eleven (11) and twenty (20) advisees. We also learned that we have a fairly large number (12) of very experienced advisors who could potentially be a resource for other newer advisors. We were a bit surprised to find that ten (10) advisors advise more than thirty (30) students, which could be high depending on the advisors’ other responsibilities. Nearly 80% of the respondents have a positive attitude toward advising but less than 60% indicated that students probably have a positive attitude about advising. These figures signify the need to collect more data from the students that may dispel the notion that students don’t appreciate the advising relationship.
By asking about their advising activities (question #6), it was apparent most advisors do have a fairly accurate and standard view of appropriate advising responsibilities. It was clear that schedule planning was not the only activity our advisors engage students in and that mentoring students is also seen as a common and important activity. Helping advisors understand the type of questioning/advising necessary to allow students to confirm or reject their choice of major is an issue the team may need to discuss to determine if this is something important for faculty advisors to engage students in.
Question #7 concerning respondents’ comfort levels using various online advising resources gave the team a great deal of information related to training needs. Many subjects were comfortable using a variety of resources, but more than 50% still do not use the University electronic advising notes system or are uncomfortable using it. About 1/3 are not familiar with the various advising handbooks. This information along with the number of respondents who admitted they need more advisor training confirms a plan for future professional advisor development.
Conclusion
Based on the results of this needs assessment, several initiatives have been developed to address advisor needs:
- A series of hands-on workshops, in computer labs, targeting advisors at various levels of experience and need have been developed. The workshops have been titled "Advising 101, 201, 301" and are designed to be open to the needs of those who attend any particular workshop. Each of these workshops is offered there different times during the fall semester and will be repeated in the spring. Twenty (20) advisors have already enrolled in one or more of these workshops.
- The team has also determined that additional data is needed to help advisors self-assess their advising performance and to provide them with data from student perceptions of their advising effectiveness. Assessment instruments are currently being developed for both purposes.
- A Web site was developed that included links to the ten (10) most valuable and important online resources for academic advisors. The Web site is linked from the "Faculty and Staff Resources" page of the College's homepage. A brief description of what is included in each resource is written in the link to assist advisors in finding the resources they are looking for.
- The College is now developing a competency-based "master advisor" program that will provide rewards and recognition for advisors who demonstrate mastery of required advising skills.
Needs assessments are valuable tools to help us direct and target our programming to achieve the best results. Diving right into a solution to a problem or the development of a program without conducting the needs assessment can be a costly mistake with mixed results.
The assessment can be simple or complex based on the purpose and available resources. Partnering with other campus representatives who have assessment training and experience can help anyone who is struggling with how to get started. It is a worthwhile endeavor that sends a message to the participants and to those who will receive the results that someone cares about the issues at hand enough to ask for ideas and suggestions. There will be more support and buy-in from the stakeholders if they know their input was solicited before the solution was developed.
Terry Musser
DUS Programs Coordinator
College of Agricultural Sciences
Penn State University
Tracy Hoover
Associate Dean of Undergraduate Education
College of Agricultural Sciences
Penn State University
Marcos Fernandez
Associate Dean of Agriculture
Director of Academic Programs in the College of Agriculture
Purdue University
Cite this using APA style as:
Musser, T., Hoover, T., & Fernandez, M. (2008). Get the Horse Before the Cart: Conducting Assessment of Advisor Development Needs. Retrieved from NACADA Clearinghouse of Academic Advising Resources website: http://www.nacada.ksu.edu/tabid/3318/articleType/ArticleView/articleId/622/article.aspx