RAILS:: Forum

recent posts - RSS

Home » Recent messages

6/8/2012 11:07:01 AM
Topic:
ACRL Assessment Discussion Group

Megan Oakleaf
Megan Oakleaf
Administrator
Posts: 31
Announcing the first meeting of the newly created ACRL Assessment
Discussion Group!

For our first meeting, we have two main topics for discussion:

* Joan Lippincott of the Coalition for Networked Information will
introduce the topic of qualitative methods in library assessment for discussion. There will be
targeted questions for you to discuss.

* Sarah Passoneau, Assistant to the Dean and Assessment Librarian,
Iowa State University, will introduce the topic of rubrics for assessment. Again, there will
be discussion questions.

* Finally, we'll save some time to brainstorm a list of topics for
future discussion.

Please join us for what promises to be an informative event:
Saturday, June 23, 2012
2:30 - 4:00 p.m.
PIER - Pacific Ballroom
No RSVP necessary. Just bring your curiosity and interest.
5/9/2012 4:01:05 PM
Topic:
Still accepting libraries/colleges in RAILS?

Megan Oakleaf
Megan Oakleaf
Administrator
Posts: 31
Hi Isabel,

Good question! I have funding to work with one more institution, but the process will be different from the one we used with the first 9 institutions. This time, we're investigating how this process works when the PI (me!) doesn't shepherd the process, especially with a campus visit. I'll be sharing more information on this in a couple weeks. Thanks for your interest!
5/8/2012 1:25:18 PM
Topic:
Still accepting libraries/colleges in RAILS?

iespinal
iespinal
Posts: 1
I'm wondering if you have room to accept more libraries into this program.

Isabel Espinal
UMass Amherst
4/6/2012 10:18:47 PM
Topic:
Writing Rubics - 2nd Green Group

erinbohman
erinbohman
Posts: 2
Brenda, you just addressed one of the questions that has been floating around in my head for some time now. ALL of my assessments will cost money, if only in staff time, but they will use considerable staff time at that. I find myself thinking that there should be a budget for the assessment plan, but this is something that I definitely don't have in my project plan at the moment. Is this something that we should consider adding?
4/6/2012 5:36:35 PM
Topic:
Writing Rubics - 2nd Green Group

Bonnie Foster
Bonnie Foster
Posts: 3
THAT is an excellent question, Brenda! What kind of assessment plan would cost money to implement? What is the need of this particular assessment plan?
4/6/2012 10:51:55 AM
Topic:
Writing Rubics - 2nd Green Group

brendasue
brendasue
Posts: 1
Paul, you make a valuable point here. Budgets will always be the nagging voice in the back of project planners mind, but they should not be an inhibitor in the rubric. Creating different rubrics, or even project plans, based on budgets is a great idea. Professor Oakleaf stated, at some point in the class, to always have an idea of what you would do with a project if you suddenly came into money. I think her words were The ones who have a plan on how they would spend the money are usually the ones that get the money. Budget constraints should not necessarily affect the outcomes of a rubric. However they may impact the assessment methods used, and therefore the data collected from these assessment methods.
Does anyone have assessment plans that cost money to implement?
edited by brendasue on 4/6/2012
4/6/2012 1:08:05 AM
Topic:
Writing Rubrics - 2nd Pink Group

kms_muse
kms_muse
Posts: 1
Possibly the most important discovery I made from this exercise is that I've been overthinking my goals and outcomes far too much! There are so many places I want to go with my ebook PDA project, it's been hard to focus on just one or two goals for which to develop a set of outcomes and supporting indicators. Creating a rubric really helped to clarify them for me. When forced to consolidate my ideas into a table, I suddenly found where I could and could not possibly hope to discover real, measurable success not only of the program for its own sake, but success in terms of changing user behaviors. For whatever reason, I hadn't really even considered that aspect of the program's impact until now.
4/6/2012 12:36:54 AM
Topic:
Writing Rubrics- Green group

alheilig
alheilig
Posts: 1
Wow, this was not easy to find. I am having a bit of trouble maneuvering through the site and an even harder time working on my rubric within the site. I think I will just have to manipulate my rubric and import it. What a great resource this is, if only I had this when I was teaching! Rubrics are a key part of evaluation and their straightforwardness I think only enhances their popularity. I had one advisor though who hated rubric because she saw them as a crutch. I think that they are only a crutch if you only rely on them all the time. We are librarians, we have to think out of the box to stay relevant! Well, I will see you girls on BB.
4/5/2012 10:36:00 PM
Topic:
Writing Rubrics- Green group

alice
alice
Posts: 1
Hi, green group! Now that I've found you, I'll add that I've also found lots of rubrics--mostly within academic assessment plans--many on the National Institute for Learning Outcomes Assessment (NILOA) website (http://www.learningoutcomesassessment.org/TransparencyFrameworkIntro.htm). In fact, NILOA's subtitle succinctly states why rubrics are important assessment tools: Making learning outcomes usable & transparent.

On their blog DeWitt (2012) questioned the percentages applied to student outcomes. "A 20% failure rate would be unacceptable in any other context" (para. 9). I, too, have wondered about percentages, only from a perspective of randomness.

What appears logical to me about rubrics is seeing a service at a glance in incremental levels of indicators and developmental degrees of progress, e.g., 45% of end users can comprehend the module, or 25% of the board can define the project. "Complex products or behaviors can be examined efficiently" (Manoa, n.d., Why use a rubric? section, para. 1).

I decided to include planning as an indicator after I spotted a rubric with planning and budgeting listed together. I also liked its level descriptions: initial, emerging, developed, and highly developed (WASC, 2007, p. 8). In her article DeWitt (2012) mentioned some as beginning, easy, practical, and inspiring (para. 6). Next, I'm off to translate rubric-ese from academic to public. - Alice

References

Dr. Patricia DeWitt. (2012, March 27). What is satisfactory performance? Measuring students and measuring programs with rubrics [Web log post, National Institute for Learning Outcomes Assessment blog]. Retrieved on April 5, 2012, from http://illinois.edu/blog/view/915/72187?count=1&ACTION=DIALOG

Manoa, University of Hawaii, (n.d.). Creating and using rubrics. Assessment How-to. Retrieved on March 31, 2012, from http://manoa.hawaii.edu/assessment/howto/rubrics.htm

WASC [Western Association of Schools and Colleges]. (2007, August 10). New tools for teams and institutions: Rubrics for evaluating the effectiveness of assessment processes. Retrieved on March 31, 2012, from http://www.csufresno.edu/oie/assessment/documents/WASC_W13_Handout1_000.pdf
4/5/2012 9:26:10 PM
Topic:
Writing Rubics - 2nd Green Group

plkandel
plkandel
Posts: 2
Seems to me like the budget is a constraint or driver of how aggressive your rubric can be, but it is not an actual part of the rubric. Maybe you can make multiple rubrics, saying "this is what we should aim for with budget x, this is what we should aim for with budget y". In fact, I have seen approaches like this from project leaders pitching new technologies to executives committees at Intel, and it often works by getting the executives to really think through the direct ramifications of budget decisions.

In general though, budget seems to me to be a category separate from overall project success. If you see that you need more money than expected to succeed, you can say "we currently stand at the low end of expectations as defined in this rubric. We need a bigger budget to move up."

bmfoster wrote:

Hiya Carlie! Your thoughts on the budget really helped me out. I'm considering backburnering (?) the budget on mine, because they don't even have the staff power to do more than just get a general increase in enrollment first. I've been overthinking this for a really long time & my plan (in it's entirety) is going to focus on just increasing enrollment and attendance and collaboration. This is only because if they can utilize this plan in the near future, the assessment of this plan can be used in an attempt to get their budget increased (it was slashed by $80k back in December). The only time budget will be mentioned is in it's cost-effectiveness from collaboration with local vendors and the community as a whole.
4/5/2012 9:17:11 PM
Topic:
Writing Rubics - 2nd Green Group

plkandel
plkandel
Posts: 2
My feeling is that budget should be an integral part of planning for results scenarios, but should not be part of the rubric. The rubric focuses on the overall goals of the project. If one of your key, measured goals of the project is cost savings, then that would count. But the budget itself seems like something we assume based on the extensive planning done elsewhere. If we get it wrong it needs to be addressed, and it may force us to revise the rubric, but it does not stand on its own as a focus of the rubric.

Does that make any sense?
edited by plkandel on 4/5/2012
4/5/2012 8:45:20 PM
Topic:
Writing Rubics - 2nd Green Group

Bonnie Foster
Bonnie Foster
Posts: 3
Ok, to Erin first... I don't know. On one hand, I think that the assessment of the marketing is necessary, but is that what is needed for this particular assessment plan? When we assess the success (seriously, say it 5x fast) of a marketing campaign at work, we are looking at how many impressions (homes) did that letter hit? Did the marketing reach homeowners? Now, in the overall assessment, it seems to be going back to the project as a whole. How many more homeowners are going to the library because of their original participation in...the summer reading program? <-- unbelievably selfish of me, but I'm trying to answer my own questions for myself

Hiya Carlie! Your thoughts on the budget really helped me out. I'm considering backburnering (?) the budget on mine, because they don't even have the staff power to do more than just get a general increase in enrollment first. I've been overthinking this for a really long time & my plan (in it's entirety) is going to focus on just increasing enrollment and attendance and collaboration. This is only because if they can utilize this plan in the near future, the assessment of this plan can be used in an attempt to get their budget increased (it was slashed by $80k back in December). The only time budget will be mentioned is in it's cost-effectiveness from collaboration with local vendors and the community as a whole.

Oh, please tell me if this is wrong!
4/5/2012 12:51:12 AM
Topic:
Pink Group thoughts on rubrics

hdsmit01
hdsmit01
Posts: 1
I too have no experience writing rubrics, but going through the other rubrics has clarified it for me. I think it will really help me in my assessment plan so as to fully and completely assess all project outcomes. It will keep me on track and evenly distributing my assessment tactics and tools.
4/4/2012 10:10:14 PM
Topic:
Pink Group thoughts on rubrics

Jim Thomas
Jim Thomas
Posts: 1
I found the rubrics posted by classmates very helpful. The indicators I had initially planned on incorporating into the rubric completely changed when I viewed other rubrics, and I thought about service measurements in new and unexpected ways.
4/4/2012 3:29:59 PM
Topic:
Pink Group thoughts on rubrics

jencelene
jencelene
Posts: 1
I've never had any experience with writing rubrics -- only with looking at them associated with grad school assignments. I appreciate the way they organize thoughts and information about services, and outline clear parameters for assessment. I like the format.
4/4/2012 2:45:01 PM
Topic:
Writing Rubics - 2nd Green Group

carliebrary
carliebrary
Posts: 2
I'm starting to have second thoughts about my previous comment.

It might be the lack of sleep talking here, but it seems to me the rubric, when applied to the entire system or service, takes into account many aspects of the plan: scalability, use, satisfaction, marketing, learning if it's applicable, staff, and yes, sometimes budget. In my case, budget doesn't really have an impact, but I could put in an indicator about staff time because if staff time is high and user satisfaction is low, then some serious changes would be required.

I also want to go all the way back to my user needs statistics to see if I can tie it into the rubric. It might be valuable to track statistics/staff anecdotes for multimedia questions asked at our desk in the rubric. The idea is that basic questions decrease and advanced questions increase, which in turn would mean the multimedia service or system would evolve, adding in more sophisticated series as users learn more.

I don't know if that would actually happen, but having an rubric to cross-reference staff anecdotes, stats, faculty impressions, and user confidence levels might give a really clearly indicate to decision makers if changes (hopefully increases!) are needed.

And I don't think I'll include all of my outcomes in the rubric...I'll talk about that more in Blackboard once I've submitted my finished rubric.

Whew, time to write my rubric, thanks for letting me talk through that one!
4/4/2012 2:07:11 PM
Topic:
Pink Group thoughts on rubrics

katburd
katburd
Posts: 1
Once upon a time, when I used to teach, I made rubrics for student assignments. I am having a harder time trying to wrap my thinking around creating rubrics for our outcomes but it is helping me think through some aspects of my plan.
4/4/2012 11:38:38 AM
Topic:
Writing Rubrics- Green group

joyferguson
joyferguson
Posts: 1
I am finding it challenging and useful to think of what success looks like from different angles. I noticed on many of the other draft rubrics there is a line for budget, which I would not have thought about. The rubrics format also provides an overview in limited categories so the project isn't just a long list of outcomes.
4/4/2012 10:21:52 AM
Topic:
Writing Rubics - 2nd Green Group

erinbohman
erinbohman
Posts: 2
I'd have to agree with Carlie - I think that a budget should probably remain a part of other planning aspects of the service rather than being included in the rubric. That is, unless a budget assessment is a part of the plan itself!

So far, i haven't tied in any of my marketing outcomes, mostly because I'm just trying to do the planning outcomes well, and adding in marketing outcomes seems to be too much for me to get done in time! But I wonder if most people are planning on tying in marketing outcomes into their rubrics and assessment plans?
4/3/2012 11:37:23 PM
Topic:
Writing Rubics - 2nd Green Group

carliebrary
carliebrary
Posts: 2
Hi Bonnie, Sorry I'm late to the party! I've been working hard on my assessment plan, like everyone I'm sure I wonder is the budget more of an input than an outcome? I would say it is very important to stay on budget, but it's not really a major component of assessment.

I agree, enrollment seems to be a common thread. For me, I'm still working through how to do this, but would like to tie in marketing and learning outcomes. So enrollment will be there, as will timing, and levels of learning (define/plan/integrate/confidence)...or at least that's what I've got right now. I'm sure it'll change!

Carlie

Home » Recent messages