Group Memory - Small Group Discussion

Assessment of Advising Interest Group Meeting

2003 NACADA National Conference in Dallas

October 3, 2003

Discussion Topic:   "What are your concerns on assessment of advising and what more can the Assessment of Advising Interest Group/Commission do?"

NEED FOR TRAINING/PROFESSIONAL DEVELOPMENT ON ASSESSMENT

Training and professional development activities needed

•  Education on the use and misuse of statistics

- problems with using inferential statistics

•  Level of expertise needed to conduct appropriate assessment.

•  Time and effort if training and development is required

- at national level and/or local level

•  Creation of a primer that allows people to get started down the right (versus

    wrong) path to assessment.

Conference materials on assessment be made available on line.

Assessment is not evaluation or research.

WHO/WHAT TO ASSESS

How to get started.

Common definition of what assessment of advising is - that bridges all levels and

institutional models of advising

Faculty resistance - in terms of their responsibility

- other side of the coin is need for advisor/program self-assessment

Assessing advisors versus programs

Cost to Benefit ratio

Reminding ourselves to be "thoughtful" in designing assessment activities (don't jump from the frying pan into the fire)

Formative versus summative assessment. 

TOOLS/INSTRUMENTS/METHODS

Decision process between ready-made tools versus developing own tool

• reliability and validity issues exist for both

• using the right ready-made tool (is it really seeking information you want)

Creation of a common assessment tool:

• Institution versus college-level versus department level - is this realistic?

• Different models of advising (e.g., centralized, decentralized, faculty only, staff only)

- The above is compounded with differences with advising philosophies

- Is it realistic considering the above?

Accounting for adequate response rates

• fatigue

•  motivation

•  length of tool

Mechanics of distribution or "sampling"

Need to use multi measures to validate information gathered.

• Use quantitative versus qualitative tools

• Use indirect versus direct measures of outcomes

OUTCOMES

Need to look beyond student satisfaction/perception to actual outcomes, with learning outcomes being the most important.

Identifying proper outcomes

Looking beyond student satisfaction

• Student self-assessment is just as important - how do they think they are doing?

Identifying appropriate learning outcomes

• should include both process as well as outcome indicators

• what is the outcome grounded in (service orientation, learning orientation, etc.)   

Advisor to student ratios influence assessment results when looking at processes and outcomes

LARGER ISSUES

Using the results of assessment in an appropriate manner.

• misuse/abuse of assessment results (improper evaluation leading to unwanted   changes versus leading to programmatic enhancement) Institutional/College

Political climate

• shoved down your throat or strategizing on how to contribute to the   development of (decision on) methods and tools that are appropriate for   needs/wants.

Ethics of assessment

Benchmarking - How? Can we? Should we?