News & Press

Data: Now What? Data Beyond High School

December 2008/January 2009 | Volume 66 | Number 4

Elliot Washor, Karen Arnold and Charles Mojkowski

Big Picture schools track graduates’ successes—and struggles—in life after high school.
Mary graduated from the Met School in Providence, Rhode Island, and entered a four-year college. During her freshman year, she ran into financial difficulties that affected her grades. She enrolled in a local community college, received an associate’s degree in nursing within two years, and obtained work in her chosen field. The following year, Mary began pursuing a second associate’s degree to advance her career. She anticipates graduating as a nurse practitioner by 2011.

Not an unusual career trajectory for a young adult in the 21st century, you might say, and
you would be right. What’s unusual is that the Met School—one of 55 high schools
throughout the United States associated with the nonprofit Big Picture Company—knew
every step that Mary took along that pathway. The Met has followed the learning and
career trajectories of many graduates since 2003; for the past two years, it has done so for
all graduates.

The Big Picture of Success
Schools affiliated with Big Picture adopt the philosophy of educating “one student at a
time within a community of learners.” Our schools are built on three basic principles: (1)
learning must be based on the interests and goals of each student, (2) curriculum must be
relevant to people and places in the real world, and (3) students’ abilities must be
measured by the quality of their work.

Students at our schools predominantly come from low-income, urban, and minority
backgrounds. Many speak a first language other than English, and few of their parents
have attended college. In short, our students are members of demographic groups that
frequently struggle to complete high school or attain postsecondary degrees. Yet in Big
Picture schools that serve grades 9–12, the high school graduation rate is 92 percent, with
95 percent of graduates gaining acceptance to college. (See www.bigpicture.org for more
information on the schools and our follow-up of alumni.)

Faculty at Big Picture schools define student success using measures beyond
standardized tests scores, grades, or even college graduation rates. We also view success
in terms of whether graduates show evidence of persistence in college, progress in their
careers, successful relationships, civic involvement, and happiness. And we measure
these life characteristics. In addition to collecting the student performance data that high
schools typically gather, Big Picture schools track our students’ trajectories for about 12 years after they graduate.

This long-term focus is unusual. In this era of heightened accountability, schools collect
more student performance data than ever. In addition to test scores, many administrators
keep tallies on student retention, college readiness, and graduation rates. But most data
collection stops at graduation, probably because schools have few resources to devote to
gathering longitudinal data, and most principals’ time is packed with using current student
data to improve practice. Learning about students’ lives a dozen years into the future is,
understandably, a low priority.

But the long term is a priority at our schools. We use a longitudinal data system to help
us understand which conditions support students’ transitions to college and careers, to
inform school improvement, and to support the continued personal and professional
development of our alumni.

What Our Alumni Database Does
Big Picture has developed an alumni manager database that connects key data on students
during their high school years to information on their lives after high school.1 This
database integrates information about students over time and enables school staff
members to centrally record data on alumni contacts and keep track of graduates’
whereabouts. While students attend Big Picture schools, school staff members enter
information about each student into the database; they also record notes about any
important conversations between that student and teachers into an “interaction log.”
But the centerpiece of this interactive database is a series of Web-based surveys designed
by our longitudinal research advisory team that we send electronically to all alumni. As
alumni respond to these surveys, they enter answers directly into the alumni manager
database. We therefore connect assembled data across information sources with a process
that requires little administrative oversight.

These surveys shed light on how graduates are transitioning from high school to work or
college, particularly in the calendar year following graduation. Thereafter, alumni enter
yearly updates on a spectrum of educational, vocational, and personal measures. Big
Picture has implemented this database in all our high schools. We provide training in
collecting data, preparing reports, and using data to guide program improvement and to
communicate key information about student outcomes to stakeholders. In addition to
collecting quantitative information about graduates through the alumni manager system,
Big Picture is helping interested schools collect qualitative data on selected students to
gain insights beyond the group data.

This longitudinal data gathering complements Big Picture’s transition program, which
helps students get ready for college and supports them as they proceed through the
difficult freshman and sophomore years—or assists students who enter the workforce
directly after high school. Transition counselors work with students beginning in
freshman year, advising them on the college admissions process and helping them plan a
solid college-going strategy or a beginning career move. Counselors keep in touch with
their assigned students throughout the summer to make sure they are on track with
pursuing the plan they developed and to intervene if necessary. They also follow these
alumni once they are in college or on the job and coordinate sending surveys to them
through the database. Once a year, all transition counselors and Big Picture principals
meet to share and discuss alumni responses.

To collect in-depth information about graduates’ lives, we use both standard surveys and
new technologies like Facebook or Twitter that engage contemporary youth and draw on
relationships between Big Picture school staff and alumni. Counselors who studied how
best to intervene with our college-bound alumni this past summer found Facebook to be
the most effective means for getting students to welcome offers of assistance. All
surveys, which students fill out online, invite students’ open-ended comments about their
lives.

Our Data-Gathering Cycle
Big Picture schools launch a yearly cycle of data collection beginning in students’ senior
year of high school. In spring, graduating seniors respond to an online survey asking
them to evaluate their high school experiences, assess their perceived readiness for
college and work, and describe their plans for life after secondary school. Students reflect
on how well specific academic and internship experiences prepared them for the future,
name their influential relationships, and—if further education is their next step—specify
why they have chosen to continue their education and selected a particular university. In
keeping with Big Picture’s broad definition of success, we also ask our about-to-graduate
seniors things like whether they feel able to name and follow their own interests and how
comfortable they feel reaching out to others for help. Response rates for the senior exit
surveys range between 43 and 94 percent.

Graduating seniors’ advisors also complete a spring survey that assesses each advisee’s
academic and personal college readiness. In late summer, transition counselors personally
contact each of the graduates they track and collect information about what each youth
actually plans to do in the fall. These contacts capture between 54 percent and 90 percent
of our just-graduated seniors. Counselors compare this information with the plan each
graduate crafted in the spring to make sure students are on track to follow their plans and
to offer assistance if necessary.

We explore how recent graduates are faring by sending a First Fall Update survey to the
most recently graduated class—both college attenders and students who did not enroll in
college—in mid-October. Using the database, Big Picture’s transition counselors send the
appropriate group of students an e-mail invitation to complete the survey and follow-up
reminders. The e-mail contains links to the informed consent form and the survey itself.
We time survey delivery to coincide with the “six-week mark” at which most collegeleaving
decisions are made (Tinto, 1993).

The survey asks about students’ paid work, probing whether it relates to their intended
career. Alumni also describe their living situations and relationships with parents, friends,
partners, and children. We ask whether they are connected to supportive adults and how
closely they communicate with their parents about academic, financial, and personal
concerns. Now that survey results are immediately accessible to high school faculty
through the database, we plan to design mechanisms to follow up with alumni who send
clear signals of struggling.

For alumni enrolled in higher education, a section asks about their experiences at college
and their use of college resources. We include questions that help us get a feel for how
students’ social and academic transitions to college are going. For instance, we ask how
challenging they find the academics, whether they have developed a personal relationship
with a faculty or staff member, and how comfortable and welcome they feel at college.
Students also complete the Non-Cognitive Questionnaire that William Sedlacek (2004)
developed as an alternative to the SAT for predicting success in college among
underrepresented students.

We’ve had some challenges ensuring that all schools keep updated contact information,
initiate surveys and reminder e-mails, and push alumni to respond. Principal buy-in is a
crucial factor in ensuring high response rates, and a systemwide coordinator is key to this
kind of endeavor. Big Picture has a national transition network coordinator who plays this
role.

Our next data-collection point is at the end of the calendar year. Big Picture high schools
that subscribe to the National Student Clearinghouse’s Student Tracker service (which
maintains college enrollment records for 91 percent of U.S. colleges and universities)
receive enrollment information for their graduates from all class years. These data give us
a comprehensive picture of how many alumni are still enrolled in college or have
received a degree.

We also send an annual survey seeking updated information from all alumni, not just
recent graduates, in hopes of learning how they are progressing on their chosen pathways
and what supports they need. This element of data collection has been challenging, as we
have not received much information from older alumni. We are investigating alternative
means for long-term follow-up, including social networking technology and reunion
events.

What Long-Term Tracking Showed Us
Findings from the first two years of collecting longitudinal data on students have
provided evidence that key elements of the Big Picture model are working well.
Graduates identify relationships with their advisors at Big Picture schools and learning
through experiential internships as central to their success. Students and advisors both
assert that the Big Picture model helped them develop motivation, resourcefulness,
independence, and communication skills. Some findings mirror national trends in urban
education. For instance, students and advisors consistently point to weak mastery of
mathematics and problems with quantitative reasoning as an obstacle to achieving postgraduation goals. This finding was no surprise, but it added strength to our impetus to
improve this area of instruction. We are now developing a quantitative reasoning
program for all our schools.

During the initial year of our longitudinal research, we were surprised to discover that
things change for many students in the summer after graduation. College plans were
complicated by financial aid concerns. We found that, generally, at least one-third of Big
Picture students seriously reconsider their college plans or change their intended college
during the summer after graduation. At least one in five decide not to begin college at all.
In 2007, for example, close to 100 percent of our seniors were accepted into college,
nearly 90 percent stated in spring that they intended to start college the next fall, but only
70 percent were actually enrolled in college come September. We realized that college
acceptance may not be enough to guarantee college access, particularly for students who
may be the first in their families to pursue higher education.

We responded to this discovery by having transition counselors keep better track of
graduates in the summer after graduation. One school, for example, employed counselors
beyond their usual 10-month contract to do summer follow-up. This summer, we will be
conducting an intervention study with our transition counselor outreach.

Worth the Cost
We believe such longitudinal data gathering should be part of every high school
accountability system. Instituting such systems will require enlightened policies,
additional resources, and training and support for principals and school faculties. But the
current high cost per graduate of such systems could be reduced through scaling up—and
we are convinced that the benefits to students will outweigh the costs.

Tracking Big Picture students has shown us that for many students, particularly lowincome
youth who in the past might not even have considered college, the traditional four
years of high school followed by four years of college is not a realistic option. Even when
academic readiness is not an issue, the social and economic barriers are formidable. The
trajectory Mary followed in the opening anecdote—tacking between learning and work as
she pursued career goals—is likely to become even more common. Schools must follow
all their “Marys” to help them achieve success in life as well as in school.

References
Sedlacek, W. E. (2004). Beyond the big test: Non-cognitive assessment in
higher education. San Francisco: Jossey-Bass.
Tinto, V. (1993). Leaving college: Rethinking the causes and cures of student
attrition. Chicago: University of Chicago Press.

Endnote
1 The James Irvine, Lumina, and Bill and Melinda Gates foundations provided support
for developing this database.

Elliot Washor is Cofounder of the Big Picture Company and the Met School in Providence, Rhode Island; 401-374-4664; ewashor@aol.com. Karen Arnold is Associate Professor of Higher Education at Boston College and Director of the Big Picture longitudinal study. Charles Mojkowski is a consultant to the Big Picture Company and the Met School; cmojkowski@mac.com.

Copyright © 2008 by Association for Supervision and Curriculum Development

Learn more about Big Picture Learning.