Follow by Email

Thursday, May 31, 2012

Foundations Series: What is Dimensions? - Topical/Conceptual Knowledge

Greetings alignment fans! I hope that all of you had a fantastic Memorial Day Weekend, and are feeling rejuvenated. I know I am. I spent the weekend with my wife and dog in Fargo, ND visiting some dear friends. Believe it or not, I didn't even think about alignment. Ok, I hardly thought about alignment. Sometimes it just jumps right in ol' noggin. So, last month, I more or less took off from blogging. Nevertheless, I'm here to dig into another foundational alignment concept. Before I get too far down that path, let's briefly recall the working definition for the term alignment:

"the extent to and how well all curricular categories and the elements within them (e.g., content standards, instructional content, and assessment practices) work together to guide instruction and, ultimately, facilitate and facilitate student learning." (e.g., Webb, 1997)

The point of this definition isn't just that alignment means that curriculum is coordinated/similar. The bigger point is that this coordination is done for the purpose of supporting student learning. If that's not our clear purpose, then our work is misguided. Ok, moving along...

The month before my brief hiatus was focused on the question "What is Directionality?" You can see the overall alignment picture for the framework I use, as well as where Directionality fits, under the green oval on the left.



This month, I will focus on the question "What is Dimensions?" You can find that in the figure above under the blue oval in the center. That question, I realize, is a bit clunky the way it is phrased. Think of it this way: the term Dimensions is a categorical label. Anyway, Dimensions is such a broad and important concept, I'm actually going to be breaking it down into three sub-categories, each of which I will explore in single blog posts. I will start with the alignment dimension of Topical/Conceptual Knowledge.

What is topical/conceptual knowledge? 

Briefly, topical/conceptual knowledge is:

the subjects, information, and ideas that students are supposed to learn. (Niebling, Roach, Rahn-Blakeslee, 2008)

A less elegant way to state that definition is this: stuff kids need to learn. Through this lens, we aren't concerned about what students are supposed to do with the "stuff" they are learning, what sort of cognitive activity we hope to evoke in their minds. No, this is really just about the facts, topics, ideas, and concepts they are supposed to learn. Here are some examples:

  • math facts
  • the Civil War
  • photosynthesis
  • author's voice
That's pretty straightforward, right? As stated, the above examples are just topics or ideas. Let's ratchet this up a notch, shall we? Let's look at some Common Core standards and see if we can pick out the topical/conceptual knowledge in them. I'll provide on English/Language Arts example and on Mathematics example. Here we go...

English/Language Arts
Standard RI.5.3. Explain the relationships or interactions between two or more individuals, events, ideas, or concepts in a historical, scientific, or technical text based on specific information in the text. 

What is/are the topical/conceptual knowledge of this standard? Think for a minute. Then scroll down and see what I think. :)



Got your answer? Here's what I thought. I believe the topical/conceptual knowledge in this standard can be found in this portion of the standard: "relationships or interactions between two or more individuals, events, ideas, or concepts in a historical, scientific, or technical text...". The standard calls on students to learn about relationships or interactions that are detailed or described in different kinds of texts. Do you agree or disagree?

Mathematics

Now, let's try a math standard:

Standard 3.MD.4. Generate measurement data by measuring lengths using rulers marked with halves and fourths of an inch. Show the data by making a line plot, where the horizontal scale is marked off in appropriate units—whole numbers, halves, or quarters.

Ok, same as before. What is/are the topical/conceptual knowledge of this standard? Think for a minute. Then scroll down and see what I think.


Got your answer? Here's what I thought. Here are the parts of the standard statement that I thought contained the topical/conceptual knowledge of the standard: "measurement data...halves and fourths of an inch...line plot...horizontal scale...appropriate units—whole numbers, halves, or quarters." In my opinion, this one wasn't quite as neat and clean as the ELA example. But I believe you can see several concepts described in the standard. There are a variety of types of measurement data (e.g., temperature, length, weight, etc.), each of which can be broken down into different units. Data can also be displayed in a variety of ways (e.g., line plot, bar graph, pie chart, etc.), again broken down into different units.

In each of these examples, we've isolated the topical/conceptual knowledge called for by the standard, but we haven't gotten into what students are supposed to be able to do with that topical/conceptual knowledge. That is critically important, but I will save that issue for next month :).

What does research say about topical/conceptual knowledge?

Sadly, we know little from research about topical/conceptual knowledge. Candidly, most research on opportunity to learn and alignment does not break down Dimensions into different categories. There really isn't any acknowledgement that Dimensions is a "thing." In my opinion, we can look to two general areas of research that can shed a little light on the subject. One area is descriptive, the other predictive.

Descriptive research

When it comes to descriptive research, we can look to work done to describe the degree of alignment between two curricular elements (e.g., intended, enacted, assessed, learned curricula). Norman Webb's (e.g., 1997) work on alignment does examine what I would consider to be a form of topical/conceptual knowledge. I would say his version of this concept is what he calls categorical concurrence. Basically, there is a high degree of categorical concurrence across standards and assessments if the same or consistent categories of content appear in both the assessment and the standards.

Andy Porter and his colleagues (e.g., Porter, 2002) have created a multi-dimensional alignment framework known as the Surveys of Enacted Curriculum (SEC). Part of the SEC framework includes a set of topical descriptors that relates quite closely to the concept of topical/conceptual knowledge. Examples of these topical descriptors includes "Linear equations" and "Main idea(s), key concepts."

Predictive research

Unfortunately, we know even less about the predictive nature of topical/conceptual knowledge than we do about it's descriptive potential. Put another way, just looking at topical/conceptual alignment hasn't been specifically explored much in research. Perhaps the most telling piece of research comes from Gamoran and his colleagues (1997) who found that just looking at topical/conceptual knowledge alignment didn't predict how well students would perform on assessments. Only when cognitive complexity was added to the analysis could those types of predictions be accurately made. We'll revisit the cognitive complexity issue next month.

What are the practical implications?

In my opinion, the practical implications boil down to a few simple ideas.

  1. It can be helpful to just think about topical/conceptual knowledge alignment first before getting into cognitive complexity. I can't provide you with empirical support for that opinion. It's based on my conversations and work with teachers and administrators who consistently tell me that thinking about cognitive complexity is harder. 
  2. Most work done under the title of "alignment" really only looks at topical/conceptual knowledge, and typically at a coarse-grained level (more to come in future blogs on this concept). Which is a decent-enough place to start, but limited. 
  3. The work gets more interesting (and challenging) when we start looking at cognitive complexity in addition to topical/conceptual knowledge.

Final Thoughts

In my experience, most folks can wrap their heads around the idea of topical/conceptual knowledge as being a "thing." For the most part, folks are also relatively comfortable looking at topical/conceptual knowledge alignment. Perhaps you fit into this description. Where there rubber meets the road, however, is with cognitive complexity. So, that is where I shall end this blog, and it is where I will pick the next alignment foundation series blog. Until then, hit me up on Twitter, and happy aligning!

References

Gamoran, A., Porter, A. C., Smithson, J. L., & White, P. A. (1997). Upgrading high school mathematics instruction: Improving learning opportunities for low-achieving, low-income youth. Educational Evaluation and Policy Analysis, 19, 325-338.

Niebling, B.C., Roach, A.T., Rahn-Blakeslee, A. (2008). Best practices in curriculum, instruction, and enacted curriculum. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology, 4(5), 1059-1072. Bethesda, MD: National Association of School Psychologists.

Porter, A C (2002) Measuring the content of instruction: Uses in research and practice Educational Researcher, 31, 3-14.

Webb, N.L. (1997). Criteria for alignment of expectations and assessments in mathematics and science education (Research Monograph No. 8). Madison, WI: National Institute for Science Education, University of Wisconsin-Madison.



Tuesday, May 1, 2012

Let's Review and Recover

Greetings lovers of curriculum alignment! I hope all is going well in your worlds as the 2011-12 school year heads towards its inevitable end. I have found myself unable to add the next installment in the Alignment Foundations series this month. I will return to that in the beginning of June. In the meantime, I'd like to share links to all of the previous blog posts in the Foundations series, as well as provide an update on the curriculum alignment work I've been doing lately.

Foundation Series Links

Below are links to each of the Curriculum Alignment Foundations Series posts. Tell your friends.

Introduction to Curriculum Alignment Foundations Series
Foundations Series: What is Curriculum Alignment?
Foundations Series: What is Curriculum?
Foundations Series: What is Intended Curriculum?
Foundations Series: What is Enacted Curriculum?
Foundations Series: What is Assessed Curriculum?
Foundations Series: What is Learned Curriculum?
Foundations Series: What is Alignment?
Foundations Series: What is Directionality?

My Recent Curriculum Alignment Work

I have been working on several things lately. First and foremost, I've been working working with an amazing programmer, Lori Thelen, at Heartland Area Education Agency 11 in Johnston, IA to add cognitive complexity tools to the Iowa Curriculum Alignment Toolkit, or ICAT for short (check out Learning Station #1). For those of you unfamiliar, the ICAT is a web-based alignment tool I started building almost four years ago for educators in Iowa to use, free of charge. The ICAT allows teachers to reflect on the content of their enacted curriculum, and to then check to see the degree to which that aligns with the intended curriculum of the Iowa Core. We will be starting a field study in the coming weeks, and hopefully have it functional by the end of June, which is when my current contract with the Iowa Department of Education ends.

Relatedly, I designed and facilitated a study determining the cognitive complexity of the Iowa Core in English/Language Arts and Mathematics for grades K-2 and all of the Iowa-specific additions to the Iowa Core for grades K-12. I say additions, because Iowa is a Common Core State Standards state. We didn't have to do a study for grades 3-12 because WestEd did a cognitive complexity study of the Common Core in English/Language Arts and Mathematics for the Smarter Balance Assessment Consortium (SBAC). That means we were able to use the cognitive complexity information from the SBAC study for our Iowa Work. I am hopefully we will be able to provide a project report on our study, as well as guidance on appropriate use for the cognitive complexity data beyond the ICAT.

Final Thoughts

I've been quite busy lately, and I'm loving every minute of it. I'll be back next month, better than ever, with a new blog post in the Curriculum Alignment Foundations Series: What Are Dimensions. Until then, be great!


Friday, April 6, 2012

Kids First, Politics...No Place Here: Don't Make 3rd Grade Retention Policy in Iowa

Despite repeated pleas from educators, it appears that the Iowa state legislature is continuing to charge forward with a policy that would include the retention of 3rd graders should they not meet criteria for reading proficiency. I should back up and say that, there are many provisions within the house and senate bills that positively promote reading skills in kindergarten through third grade. You can find the latest versions of those bills here:

House File 2380
Senate File 2284

Despite an overwhelming large set of evidence indicating the long-term negative impact of grade retention, the Iowa state legislature continues to charge forward with this provision. Of course, not all legislators are in favor of this provision, but enough are as of the last time votes were counted to pass some version of a bill that would include a 3rd grade retention provision.

Many others have quite eloquently argued against this provision, and I don't want to repeat what others have said. At least, not in the same way. Scott McLeod, for example, just posted a blog on this subject. I posted a response to Linda Fandel's (The Governor's Special Assistant for Education) blog entitled "If a child can't read in fourth grade" laying out my concerns about the 3rd grade retention proposals. Which, I should  note, she never responded to. Granted, I'm sure the Governor's Special Assistant is very busy. But then again, aren't we all? You can read my response to her post here.

As I said, I don't want to rehash either what I've said, or to write up thoughts that have been posted elsewhere. Instead, I am sharing links to just a few of the many resources that refute the idea that codifying grade retention as state policy is not founded the evidence available. I am not posting everything I come across. Here's your challenge: do a Google search using the phrase "research on grade retention" and see what you come up with. That's what I did. Here we go, in no particular order:

ASCD, Educational Leadership: http://www.ascd.org/publications/educational-leadership/mar08/vol65/num06/Grade-Retention.aspx

National Association of School Psychologists: http://www.nasponline.org/resources/handouts/revisedpdfs/graderetention.pdfhttp://www.nasponline.org/resources/instruction_curriculum/retentionho_educators.pdf

Dr. Shane Jimerson's "Beyond Grade Retention and Social Promotion (several links to meta-analyses and empirical studies on this page): http://www.education.ucsb.edu/jimerson/retention/

Advocates for Children: http://www.advocatesforchildren.org/pubs/retention.html

Center for Child and Family Policy-Duke University: http://www.childandfamilypolicy.duke.edu/pdfs/pubpres/FlawedStrategy_PartThree.pdf

Regional Educational Laboratory Northeast and Islands: http://www.relnei.org/documents/RDR_2010_01_08_Grade_Retention_and_Dropout_Rates.pdf

Wrightslaw: http://www.wrightslaw.com/info/retain.index.htm

Lorrie Shepard's Research on Grade Retention: http://www.colorado.edu/education/faculty/lorrieshepard/retention.html

The National Center for Fair and Open Testing: http://fairtest.org/grade-retention-still-failed-policy

Center for Mental Health in Schools: http://smhp.psych.ucla.edu/pdfdocs/briefs/retention.pdf

Journal of Applied School Psychology: http://www.tandfonline.com/doi/full/10.1080/15377903.2010.518843

National Institutes of Health: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2864494/http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2329586/

Center for Local, State, and Urban Policy: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.184.2180&rep=rep1&type=pdf


To be fair, it took some digging, but I did find some resources that promote the idea that grade retention can have positive benefits, or at least indicate it may not be as harmful as some claim. Here are some of the few examples I found:

Education and Finance Policy: http://www.mitpressjournals.org/doi/pdf/10.1162/edfp.2007.2.4.319

National Institutes of Health: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2921809/pdf/nihms162989.pdfhttp://www.ncbi.nlm.nih.gov/pmc/articles/PMC2901899/pdf/nihms193775.pdf


As I have said, repeatedly, putting 3rd grade retention into law is at best ill-informed, at worst significantly detrimental to at least the social-emotional well being of children, if not their academic performance as well. I sincerely hope that wiser heads prevail in this battle in the Iowa state legislature. This isn't about grownups folks. It's about kids. Let's keep that in mind.

Response to Fandel's Promotion of 3rd Grade Retention

I should have done this earlier, but I suppose I was a bit misguided in my optimism that Linda Fandel, The Governor's Special Assistant for Education, would respond to my commentary on her blog entitled "If a child can't read in forth grade." Sadly, she did not respond.

As a professional educator with some background in determining the quality of research, I felt compelled to share some of my thinking on this matter. Below is my verbatim response to Fandel. Enjoy!



Linda, I completely support the notion that students who do not display sufficient reading skills by 3rd grade have a high likelihood of struggling in many areas of their life during and beyond school. I’m excited that as a state, we have decided to shine a bright light on this issue. Furthermore, I am not a supporter of “social promotion.” By social promotion, I mean allowing students to move to the next grade level because it is the next stop in the school process, who are missing critically important skills that are needed to be successful at that next stop. It is my hope that another one of the state’s proposals, competency-based education, will help us reshape what those next steps in the school process are for our students. In that regard, I believe the current system’s structures that encourage moving students through the levels of school is flawed, especially for struggling learners.


With that said, I couldn’t be more opposed to the recommendation to retain 3rd graders who don’t demonstrate sufficient reading skills. Scott McLeod and Katie Kauffman, to date, have provided both solid rationale and links to resources to refute both the practice of grade retention and your subsequently-provided evidence that grade retention, as currently proposed, is appropriate policy. It is my experience that disagreements like these are sometimes due to the parties not always talking about the same thing. Maybe that is what is happening here.


So this is what I’m curious about, in the spirit of trying to have clear communication of ideas, beliefs, and evidence in this debate. What criteria are you (and others whom you’ve worked with to develop and promote the idea of grade retention as policy) using to select evidence for your argument? I believe I should have to answer this question as well, so let me start.


When determining what is evidence-based practice and what is not, I believe that the issue is more of a continuum than a yes/no proposition. That is, any particular practice could have absolutely no evidence for having a positive impact on student learning all the way to virtually undeniable evidence, and anywhere in between. If we can get behind this notion (which I do), then the next step for me is to have some criteria for determining where on that continuum the available evidence would suggest that practice should be. Some pretty smart people have developed these sort of criteria. Some examples include:


The Wing Institute: http://winginstitute.org/Evidence-Based-Education/Standards-of-Proof/


The What Works Clearinghouse: http://ies.ed.gov/ncee/wwc/ReviewProcess.aspx


Best Evidence Encyclopedia: http://www.bestevidence.org/methods/methods.htm


There are many other organizations and evidence-review processes that could be applied to the 3rd grade retention issue. The reality is that practices such as grade retention are rarely if ever examined using such frameworks, even by researchers themselves. As such, I tend to rest my evidence arguments on a foundation of a subset of practices that typically fall under more stringent frameworks:
-research gathered and analyzed by professional researchers and evaluators with a history of quality work
-replication of findings from previous research
-lack of replicated, contradictory evidence
-findings indicate both a statistical and practical positive impact for students



Applying either the more-or less-stringent criteria I laid out above, grade retention continues to, at best, have a neutral long-term impact on student outcomes. At worst, retained students tend to see long-team decreases in their learning and negative social-emotional outcomes. In other words, the evidence for grade retention, in my opinion, falls much closer to the “no evidence at all” end of the evidence continuum that that “almost undeniable” end by a fair margin. Here is a resource I think do a nice job summarizing these phenomena:
http://www.nasponline.org/about_nasp/positionpapers/StudentGradeRetention.pdf
Here is another resource that provides evidence-based alternatives to grade retention:
http://www.nasponline.org/resources/principals/Retention%20WEB.pdf


Do I think my one or two resources are a sufficient argument against the proposal to retain 3rd graders in Iowa that have insufficient reading skills? No, I do not. No more than your two resources provide ample evidence to support 3rd grade retention of these students. The difference, in my opinion, is that the resources I’ve provided here are representative of the larger body of evidence, applying a reasonable approach to defining adequate evidence.


With that said Linda, please, provide the evidence criteria/approach used by those who have developed the 3rd grade retention policy recommendation. I love being proven wrong about what’s good for students. That way, I learn how to be a better, more responsible educator. I think our students deserve that. I believe that we are better together, so let’s find a way to promote reading skills for all of our students that is based on a solid evidential foundation. What say you?

Friday, March 30, 2012

Foundations Series: What is Directionality?

(c) Brad Niebling
As I sit here, excited to start my next installment in the Foundations Series, I am looking outside at the beautiful weather, and feeling a bit torn. As much as I love a good blog post on curriculum alignment, it sure would be nice to be outside hiking somewhere rugged with my dog, Buddy. See, he's excited to see the world, too!

I'm guessing many of you are feeling the same way, either returning from or getting ready to go on spring break. Sometimes, we just have to find a way to push on. I'm sort of feeling that way today. But I will push on!

Last month, I kicked off the next "chapter" in the Foundations Series by exploring the question "What is Alignment?" As a reminder, I defined alignment as

"the extent to and how well all policy elements work together to guide instruction and, ultimately, facilitate and enhance student learning." (Webb, 1997)

Alignment is not necessarily a "thing" unto itself, but rather a characteristic of the relationship between different aspects of curriculum. The desire to have the different aspects of curriculum coordinated or matched makes practical sense. In my experience, where the breakdown occurs is how we view alignment. Let me ask you a question: have you ever heard a phrase that sounds something like "our curriculum is aligned"? Perhaps you've even used a phrase like that yourself. It's understandable. I hope, after reading my series of blogs defining curriculum as having many different types, that you we see alignment as more than a "yes/no" sort of thing. I can tell you that beyond that, alignment gets to be a broader thing when you break it down into smaller chunks as well.

This month, I start on a journey to break alignment down into smaller, more meaningful chunks, just like I did with curriculum. By doing so, one step at a time, I hope that together we can gain a deeper appreciation for alignment and learn how to use it more effectively. Visually, a multi-dimensional view of alignment looks something like this:


As you can see, alignment is not one single "thing." Right now, I will start from the far left of the diagram and explore the question "What is Directionality?" and work my way from left to right in the coming months. Let's get into it!

What is directionality?


That's a very good question, if I do say so myself! Wait, haven't I seen that picture before? Anyway, when I use this term in my work with folks, I mostly get blank stares. That's ok, because it's not a term that gets used very often. In general, directionality is...

"the direction in which alignment is examined." (Niebling, 2008, cheap plug)

Well, ok. That's not really helpful at all by itself, and probably not worth a citation. I think we can all wrap our heads around what the concept of direction means. Come on alignment boy, you gotta bring more game than that! Fine, you want it, you got it. Horizontal alignment is the

"degree of match, typically across two curricular categories (e.g., instructional content with state or national standards) within a single level (e.g., same grade comparisons)."

Boom goes the dynamite! I've got more, too. Vertical alignment is the

"degree of match within one curricular category (e.g., district benchmark assessments) across multiple levels (e.g., across grade levels)."

I don't even have to say it, do I?
image source - http://knowyourmeme.com/memes/boom-goes-the-dynamite

Whew! That was a lot of work, blowing up all that alignment dynamite. However, beyond me entertaining myself, we need to put some meet on that directionality bones. Let's start with horizontal alignment. When you hear or see the term "alignment," most likely it's about horizontal alignment. Every example of alignment I have discussed in previous blog posts in this Foundations series was an example of horizontal alignment. Consider this scenario:

A 5th grade teacher wants to know what the degree of alignment is between what she taught and the statewide accountability test her students took during the Spring semester.

In this scenario, the teacher is curious about the degree of horizontal alignment between her enacted and the assessed curriculum. It's horizontal because she's curious about the degree of alignment between her 5th grade enacted curriculum and the 5th grade state test. As you can see, she's focused on 5th grade only. If you look back at the definition for horizontal alignment, you'll see this scenario meets all of the criteria defined.

Now let's consider vertical alignment, which is sometimes referred to as vertical articulation in curriculum circles, or vertical scaling in assessment circles. Each of these concepts are slightly different in some ways, but the terms often get used generically. I'm not going to go too deeply into the nuances of those differences here. What I will say is that vertical alignment still calls for coordination of curricular elements to facilitate and enhance student learning. In this case, that coordination occurs within single curricular elements, not across them. Consider this scenario:

The curriculum director of a school district wants to know how well common formative assessments are coordinated with each other as students transition from elementary to middle school, and from middle to high school.

In this scenario, the curriculum director is curious about the degree of vertical alignment between assessed curriculum across grade levels. So, her focus is on one curricular area (i.e., assessed curriculum) across grade levels. Take a peak back at the definition for vertical alignment, and you should see that this scenario meets all of those criteria.

Ok, those were sort of easy. Now it's time for a tougher scenario. You decide if this scenario is about (a) horizontal alignment, (b) vertical alignment, (c) both, or (d) neither. Here we go...

Arianna Miles is the principal of North County Elementary School. There are a lot of students in her building. Each grade level has six sections. It is a school of high poverty, with 70% of students eligible for free and reduced lunch. It is highly diverse with respect to culture and ethnicity. She knows how important it is to make sure all students have equity in their opportunity to learn the intended curriculum. She wants to know the extent to which teachers' enacted curriculum is aligned with each other at each grade level.

What do you think? Here's what I think. Although you could possibly make compelling arguments for their being vertical alignment issues embedded in Ms. Miles focus, in my opinion this scenario is primarily about horizontal alignment. The tricky part is that she is curious about alignment among the enacted curriculum of multiple teachers within a grade levels. If you look closely at the definition for horizontal alignment, it indicates that this aspect of directionality is generally about examining alignment among multiple curricular elements, but not always. It is very possible to examine the degree to which what different teachers teach is the same.

What research says about directionality and practical implications

Generally speaking, the research out their on alignment isn't typically framed around directionality. That doesn't mean that directionality isn't part of the research though. For example, research studies like Cohen's and Gamoran's incorporate horizontal alignment, with the idea that as horizontal alignment increases, so to does student achievement. Vertical alignment research often comes in a very empirical package, like determining the psychometric soundness of assessment tools, or more narrative and descriptive packages, like the degree to which standards expand and require more complex thinking across grade levels. Indeed, a simple Google search for vertical articulation or alignment yield very little information, other than a few PowerPoint presentations and long documents within which vertical alignment was mentioned but not really highlighted.

What can we take from all of this? Here are the practical implications of Directionality, in no particular order:

  • A high degree of horizontal alignment is generally a good thing.
  • A lower degree of vertical alignment is generally the goal. After all, we don't want the exact same things taught and assessed at every grade level, right? 
  • Achieving these goals will take district-wide vision and a lot of collaboration.
Final thoughts

Addressing the Directionality of alignment, be it horizontal or vertical, is important to understand what the work is about and what we want to accomplish with it. When you read something about alignment, or listen to someone talking about it, make sure to inquire about the direction they are talking about. 

As always, thanks for reading. Check back next month, as I dig into the question "What are Dimension?" See you then and there!

Resources Used

Cohen, S. A. (1987). Instructional alignment: Searching for a magic bullet. Educational Researcher, 16, 16-20.

Gamoran, A., Porter, A.C., Smithson, J.L., & White, P.A. (1997). Upgrading high school mathematics instruction: Improving learning opportunities for low-achieving, low-income youth. Educational Evaluation and Policy Analysis, 19, 325-338.

Kolen, M.J. (2004). Linking assessments: Concept and history. Applied Psychological Measurement, 28, 219-226.

Niebling, B.C., Roach, A.T., & Rahn-Blakeslee, A. (2008). Best practices in curriculum, instruction, and assessment alignment. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology, (4)5, 1059-1072. Bethesda, MD: National Association of School Psychologists.





Tuesday, February 28, 2012

Foundations Series: What is Alignment?

Image Source - http://leisuredive.com
Greetings blogosphere! Guess what? It's February during a leap year. You know what that means, right? Do you? It means a whole extra day to talk about curriculum alignment! Let's get to it!

Last month, I wrapped up the first part of the Foundations Series on curriculum alignment by examining the question "What is Learned Curriculum?" Briefly, I examined what it is students actually learn in the context of the intended, enacted, and assessed curricula, after having both posed and examined the questions "What is Curriculum Alignment?" and "What is Curriculum?" Visually, I have represented curriculum as a learning-centered triangle"


I am starting with the next "chapter" in the Foundations Series this month where I examine various aspects of alignment. I am starting with a broad question "What is Alignment?" In the coming months, I'll be dissecting alignment into smaller parts, just like I did with curriculum. Before I get into alignment though, I just want to take a small bird walk and share what I'm up to right now...

Image Source - http://www.ets.org/praxis/nasp 
I'm writing my latest bog post in the Foundations Series from Philadelphia, PA. I'm a long way from Urbandale, IA, both figuratively and literally! I'm at the annual conference of the National Association of School Psychologists (NASP). I try to make it here every 3 years or so. Even though my degree from graduate school is in educational psychology, and I did my thesis and dissertation on curriculum alignment (cheap plug), I was in the school psychology program at the University of Wisconsin-Madison. So, these are my peeps. Or at least my original peeps.
Image Source - http://nasponline.org

I've been both honored and blessed to work with educators in many different disciplines over the years, and I feel like I've learned a great deal from many of those educators. In that spirit, this conference is a reminder to me of two big things that I think directly apply to curriculum alignment. First, curriculum alignment isn't just the responsibility of the classroom teacher, or the "curriculum adoption" committee. It takes entire systems to ensure tight alignment; administrators, psychologists, occupational therapists, and the students themselves, just to name a few. Second, I am reminded that there are so many other pieces in the schooling formula that impact student learning; social-emotional well-being, connectedness, reliability and validity of assessment processes and tools, and the evidence-base for instructional materials/practices, again just to name a few. So while I believe curriculum alignment is central to the success of any school system, let's not forget how many things need to be going well for a student to have a positive school experience. With all of that said, let's dig into some curriculum alignment, shall we?

Although defining curriculum was a complex process, much of what curriculum is can be framed as something tangible. Assessment materials, content standards, instructional materials, lesson plans, test results. That's not to say that there aren't more intangible aspects to curriculum, like conversations and thoughts that certainly make up part of the enacted curriculum. But there are definitely concrete curricular elements that we can put our eyes and hands on. Alignment, on the other hand, isn't really a tangible thing. Instead, it's the nature of the relationship among curricular elements. Here's an official type of definition:

Curriculum alignment is "the extent to and how well how all policy elements work together to guide instruction and, ultimately, facilitate and enhance student learning." (Webb, 1997).

This is the sort of definition that is usually more useful for researchers than practitioners. But it absolutely applies to practitioners, day in and day out. Think of "policy elements" as the intended, enacted, and assessed curricula. The "working together" part is really getting at how similar are those elements to each other. For example, if a teacher provides instruction on vowel teams (enacted curriculum), an assessment experience (assessed curriculum) that is "aligned" with that would also include vowel teams. That would be the two elements working together. If the assessed curriculum was composed of vowel-consonant teams instead of just vowel teams, that would be an example of the curricular elements not working together.

While I hope that example provides a simple picture of what alignment is, it by no means does it justice. In reality, alignment is a multi-dimensional thing. Just like the big idea of "curriculum" is composed of smaller components (i.e., intended, enacted, assessed, and learned), so to is alignment. Below is a visual of the multi-dimensional nature of alignment:


We will be digging into each an every aspect of the diagram above in the coming months. I will make the argument that unless each of these elements is considered, any alignment work will be short of getting the "bang for the buck" that alignment can bring. That's it for now. Check back next month! In the meantime, hit me up on Twitter.

References

Webb, N.L. (1997). Criteria for alignment of expectations and assessments in mathematics and science education (Research Monograph No. 8). Madison, WI: National Institute for Science Education, University of Wisconsin-Madison.

Monday, January 30, 2012

Foundations Series: "What is Learned Curriculum?"

Image source -
http://ryangoslingdisneylandcats.tumblr.com
And we're back!!! Welcome to my curriculum alignment blog. I'm particularly excited about this post. What, what was that? Why am I so particularly excited about this post you ask? Well let me tell you why. My last few blog posts in the Foundations Series have dealt with three areas of curriculum, intended, enacted, and assessed, that are often part of alignment discussions. While each of these areas, and how tightly they are aligned with each other, is an important part of successful schools, it is the Learned Curriculum that is the actual goal of the schooling process.

Let's take a step back and set the stage for this discussion. As I have indicated in the previous Foundation Series posts, the framework I use to discuss curriculum alignment is based on the work of Andy Porter. Here is a visual depiction of that framework:


Porter, in his article on curriculum assessment (i.e., collecting and using data on different aspects of curriculum, including alignment), does not actually define the learned curriculum or dig into it. I think, however, we can come up with a pretty decent idea about what the learned curriculum is. In this framework, I define the learned curriculum, sometimes referred to as the achieved curriculum, as:

the knowledge and skills acquired by students during the schooling process.

(c) Brad Niebling, Midwest Instructional
Leadership Council
There you have it. I told you we could do it! But notice what is and is not a part of this definition. Absent from this definition are how the learning relates to the intended curriculum, attributing the learning to teachers or anything they said or assigned (part of the enacted curriculum), or that students actually demonstrated the knowledge and skills they acquired and/or a score/grade (the assessed curriculum). So, students might acquire knowledge and skills as a part of the schooling process (i.e., learn), but it may not be tied to any of the other three curricular components in framework. Let's examine each one of these components and how they could or should relate to the learned curriculum.

The Intended and Learned Curriculum


Briefly, the intended curriculum is what students are supposed to learn. In this framework, the learned curriculum can take the form of content standards, curriculum frameworks, curriculum maps, textbooks and related materials, to name a few. In well-functioning systems, the intended curriculum as actually written down, and staff have professional conversations to develop a common understanding of what contents of the intended curriculum mean.

(c) Brad Niebling, Midwest Instructional
Leadership Council
As I have detailed previously, we don't always do a great job focusing on a common intended curriculum, much less integrate that into the enacted or assessed curricula. I would say we spend even less time relating student learning back to the intended curriculum. To be fair, it's not like this sort of thing is often addressed in training programs, nor is it really expected in our schools. Here is an important question to consider: how can we determine if there is any relationship between something like the Common Core State Standards and student learning if we don't explicitly analyze the alignment between the intended and learned curriculum? I realize we can do some very general correlational work, but in my mind this is short of where we need to be.

I believe that organizing student learning according to the intended curriculum can greatly enhance our understanding of student strengths and weaknesses. There are likely several methods and tools out there to do this. For example, I believe standards-based assessment and reporting has a lot of promise to assist us with linking the intended and learned curricula. Iowa is starting to explore competency-based education as a system as well. I think having a structure such as these can greatly enhance teachers' ability to organize student learning around the intended curriculum.

The Enacted and Learned Curriculum

The enacted curriculum is what students get the chance to learn, as well as how that "what" is delivered. Recall, when it comes to curriculum alignment, we are only talking about the "what" of enacted curriculum. Also note, the enacted curriculum isn't necessarily just what teachers say and do. Students own engagement, reflection, research, and other activities can constitute part of the enacted curriculum if it's related to the schooling process.

(c) Brad Niebling, Midwest Instructional
Leadership Council
Regardless of whether or not the enacted curriculum is tightly aligned with the intended curriculum, we still want to know if students are learning the things they have an opportunity to learn. Otherwise, whats the point of school? Why would teachers even show up? I'm guessing almost all teachers want to know if what they are teaching is being learned. Historically, we haven't typically explored the nature of this alignment, or lack thereof. Instead, some curriculum is enacted, another assessed, and what is learned is rarely checked back against what was taught. In my opinion, that occurs because teachers don't have good, easy-to-get enacted curriculum data to examine.

However, if we want to be able to link enacted curriculum, at least the part delivered, structured, or otherwise observed by the teacher, to the learned curriculum, we need data on both, and the extent to which they are aligned. This can be particularly useful in those cases that there are intended-learned alignment data available, and you are trying to troubleshoot why students are struggling to learn particular aspects of the intended curriculum.

The Assessed and Learned Curriculum

The assessed curriculum is a system of processes and tools that are used to determine the extent to which students are acquiring or have acquired knowledge and skills (Niebling, et al., 2008). Ideally, the assessed curriculum is tightly aligned with the intended and enacted curricula. In my experience, I've seen a lot of assumptions around the assessed curriculum, mainly that it is aligned to anything.

(c) Brad Niebling, Midwest Instructional
Leadership Council
Large-scale accountability and research assessments often have reliable and valid alignment data, typically between the intended and assessed curricula. I have seen little-to-no alignment data for most assessments that can be bought on the market. Some do, some don't. Unfortunately, I don't see alignment data from district- or teacher-created assessments. I think this is due to multiple factors. First, doing rigorous alignment work is not the norm in our schools Second, I don't find many teachers who have collectively examined the intended curriculum well enough to construct tightly aligned assessments. I'm not necessarily blaming teachers for this, it's just my observation.

Without tight alignment of the assessed curriculum to either the intended or enacted curricula, we have no way of determining if a student's learning has anything to do with the schooling process (i.e., the learned curriculum). The assessed curriculum, in many ways, is really the bridge between intended/enacted and the learned curricula.

Practical Implications

I believe that there are at least three major, practical implications to the information I just shared. Here they are, in no particular order:
  1. School is about learning: School is about many things, not just academic achievement. That debate is for another blog post. If we can agree that student achievement/learning is central to the purpose of school, then I assume that we want to know if the things they are learning are related to anything we are supposed to teach or what they get an opportunity to learn. If so, then it makes sense to think about learning in a curricular context.
  2. It's really a pyramid, not a triangle: In a well-functioning system, the intended, enacted, and assessed curricula are all tightly aligned. The purpose of this tight alignment is not really and end unto itself, but rather a means to an end, mainly that students learn. 
  3. Troubleshoot when needed: If student learning is not where it needs to be, especially when viewed as a system, how do we know why that is the case? Having alignment data among the intended, enacted, and assessed curricula can provide information on, for example, a lack of sufficient opportunity to learn that coincides with specific areas in which groups of students are struggling.
Final Thoughts on Curriculum...For Now

Over the last 6 months, I have laid out my ideas about how to define/describe the concept of curriculum. Hopefully it's been at least somewhat practical. Here are a few summary points I'd like to make before moving on to the next part of the Foundation Series, again in no particular order:
  1. Curriculum IS NOT just textbooks!
  2. Curriculum is not a single "thing." No matter how you frame it, it's important to look at it from several angles because in practice, it's complicated.
  3. Curriculum is what students are supposed to learn, what they get the opportunity to learn, what gets assessed, and what is actually learned.
  4. Ultimately, it's all about the learned curriculum. The other three areas need to be coordinated to facilitate and enhance student learning.
Image source -
http://www.shirttrader.com/shirt/i-love-lamp.html
Next, I'll be turning my attention to the next part of the curriculum alignment world, namely alignment. I love alignment! Make sure to tune in next month, when I start with the question "What is Alignment?" In the following months, I will be taking on the alignment topics of Directionality, Dimensions, and Level of Analysis before I pull it all back together. 

Not sure what those things are or why you should care about them? Then check back in next month! And keep checking back to this blog, and follow me on Twitter. Soon I will be posting a blog on the curriculum alignment implications of Iowa Governor Branstad's education legislative recommendations, and I might even chime in on the hot topic of 3rd grade retention. Sparks are flying! Thanks for taking the time to read my blog, and definitely post your thoughts or reflections in the comment section below. Tell your friends!