Education Matters: The challenge of interpreting test scores

Deciphering California’s recently posted school test scores is exasperatingly complex. It’s a dizzying alphabet soup of acronyms, numbers that rely upon other numbers, links back to other sources, and nuanced interpretations of the data.

Marsha Sutton

(Copyright of Marsha Sutton)

Perhaps the best place to start is with the Smarter Balanced tests that were administered last spring in 2022 to students in grades 3 through 8, and grade 11. Testing was canceled in 2020 because of the pandemic and was limited in 2021.

Because of this gap, educators caution not to compare the 2019 and 2022 scores.

Some educators, including San Dieguito Union High School District Associate Superintendent of Educational Services Bryan Marcus, suggest the 2022 scores should be viewed as a new baseline.

However, others advise against it, saying schools were still impacted by health-related absences and other lingering effects of the pandemic.

The goal of Smarter Balanced tests (also known as SBAC, which stands for Smarter Balanced Assessment Consortium) is to determine how well students have mastered California’s Common Core State Standards (CCSS) for English language arts/literacy (ELA) and mathematics.

The California Department of Education (CDE) defines SBAC as a group of states that brought teachers, administrators and experts together to develop tests to measure how well students understand and are able to apply the skills and knowledge required by state standards.

Since 2010, according to the CDE, multiple states have adopted the same standards for ELA and math, which provides continuity when students change schools or move to a different state in the consortium.

The SBAC developed assessments to be used by member states. California uses the California Assessment of Student Performance and Progress (CAASPP), formerly known as STAR (Standardized Testing and Reporting).

The state’s CAASPP system primarily utilizes the Smarter Balanced tests for ELA and math.

Dashboard

The Dashboard became California’s public school accountability system in 2017 (the program was suspended in 2020 and 2021) and addresses the requirements of a federal program called Every Student Succeeds Act (ESSA). It uses multiple indicators to measure student performance and progress.

Besides academics, the Dashboard offers data on suspension rates, graduation rates, chronic absenteeism and English learner progress.

For academic performance, the Dashboard relies on two factors: a school’s participation rate (which must be a minimum of 95% or a penalty is applied) and its overall Distance From Standard (DFS).

The DFS is the distance between a student’s test score and the state’s calculated lowest standard met.

A board presentation given on Feb. 13 by Bryan Marcus (go to: bit.ly/3Fjl6E5) offers an example of a student who scores 2420 on ELA and has a -12 DFS because that is 12 points below 2432 which the state has set as the lowest possible score for “Standards Met.”

A student who scores 2450 would have a +18 DFS, which is 18 points above the 2432 number.

To calculate an entire school’s ELA and math scores, the scores of all students taking the tests would be added together and divided by the total number of students.

If the school’s participation rate falls below 95%, there’s another acronym to consider.

A Lowest Obtainable Scale Score (LOSS) is assigned to each student needed to bring the participation rate up to 95%, and the LOSS for each student who did not test is used in calculating the DFS, according to the CDE.

A complicated formula was devised by the state [https://www.cde.ca.gov/ta/ac/cm/documents/changesacademicpart.pdf] to calculate each school’s DFS, using the LOSS number if a participation rate penalty has been triggered.

“The state publishes the state test scores, calculates the DFS scores, assigns the penalty, and produces all of the Dashboard reports,” Marcus said in an email.

San Dieguito

San Dieguito’s test scores reveal some major disparities among its four traditional high schools.

Although SDUHSD educates students in grades 7-12, I focused on grade 11 – not just to simplify the dissection of mountains of data but mostly because it’s useful to know how well students approaching graduation are prepared for life after high school.

The percent meeting or exceeding standards on Smarter Balanced tests for SDUHSD high schools shows the following results compared to 2019, according to a report by CalMatters. Visit https://calmatters.org/education/k-12-education/2022/10/california-student-test-scores-dive-district-lookup/

La Costa Canyon –

ELA -- 57.1% in 2022, 70.6% in 2019 (13.5% decline)

Math – 38.5% in 2022, 52.1% in 2019 (13.6% decline)

San Dieguito Academy –

ELA – 69.7% in 2022, 70.7% in 2019 (1.0% decline)

Math – 51.8% in 2022, 55.1% in 2019 (3.4% decline)

Torrey Pines –

ELA – 74.4% in 2022, 77.8% in 2019 (3.4% decline)

Math – 53.7% in 2022, 62.4% in 2019 (8.7% decline)

Canyon Crest Academy –

ELA – 88.8% in 2022, 87.9% in 2019 (1.0% rise)

Math – 79.6% in 2022, 80.4% in 2019 (0.8% decline)

The CDE shows the following participation rates in 2022 for each school:

LCC: 82% for ELA and 61% for math

SDA: 87% for ELA and 87% for math

TP: 98% for ELA and 97% for math

CCA: 98% for ELA and 97% for math

Only Torrey Pines and Canyon Crest met the 95% participation rate minimum. Because SDA and LCC did not meet that 95%, the schools were penalized.

Academic performance

The 2022 Dashboard for each school [www.caschooldashboard.org/reports] reflects these scores, which are affected by a participation rate penalty if applicable (SED stands for socio-economically disadvantaged):

LCC dashboard (total enrollment: 1,647) –

ELA – 23.7 points below standard

Math – 122 points below standard

SED – 23.3%

SDA dashboard (total enrollment: 2,145) –

ELA – 20.3 points above standard

Math – 30.2 points below standard

SED – 25.7%

TP dashboard (total enrollment: 2,649) –

ELA – 66.3 points above standard

Math – 9.4 points above standard

SED – 18%

CCA dashboard (total enrollment: 2,346) –

ELA – 126.9 points above standard

Math – 106.7 points above standard

SED – 11.7%

What stands out are LCC’s test numbers. (Also curious is LCC’s low overall enrollment number compared to the other three high schools, but that’s a different issue.)

Not meeting the 95% participation rate can hammer a school’s performance numbers, making it more difficult to indicate actual achievement.

“If this happens, performance at our schools can appear artificially low,” Marcus said.

The CDE has a detailed table [https://www.cde.ca.gov/ta/ac/cm/acaddatafiles.asp] that shows, among other statistics, the scores for students who actually took the tests, which offers a different picture of a school’s academic performance (i.e., before the penalty has been applied).

Dashboard status levels range from Very High, High, Medium, Low and Very Low [https://www.caschooldashboard.org/].

LCC’s Dashboard for ELA with the penalty is 23.7 points below standard. Without the penalty, it’s 17.7 points above standard. This places LCC in the category of Low with the penalty and Medium without the penalty.

LCC’s Dashboard for math with the penalty is 122 points below standard. Without the penalty, it’s 52.1 points below standard. This places LCC in the Very Low category with the penalty and Medium without the penalty.

San Dieguito Academy also did not meet the 95% participation rate.

SDA’s ELA Dashboard score is 20.3 points above standard with the penalty. Without the penalty, it’s 45.2 points above standard. This places SDA in the Medium category with the penalty and the High category without the penalty.

SDA’s Dashboard score for math is 30.2 points below standard with the penalty. Without the penalty, it’s 10.1 points below standard. This places SDA in the Medium category both with and without the penalty.

For the other two schools, ELA and math scores for Torrey Pines are at the High level, and ELA and math scores for Canyon Crest are at the Very High level.

Disparities

There may be good reasons why there’s such disparity in academic performance among schools, ignoring for the moment the participation rate issue. LCC and SDA have more SED students, for example.

But it’s clear that not all SDUHSD schools are created equal. Maybe they shouldn’t be – each having its own culture and unique demographics. And one day of testing is just a snapshot of overall academic ability.

With many factors to consider in evaluating a school’s success, reviewing test scores may not be the best way to assess achievement.

But it is one important indicator.

Opinion columnist and education writer Marsha Sutton can be reached at suttonmarsha@gmail.com.

Marsha Sutton is a columnist and presents her opinion. If you disagree or agree with her opinion, we’d like to hear from you. Email your comment to editor@delmartimes.net.

Column: Combines reporting, storytelling and commentary to make a point. Unlike reporters, columnists are allowed to include their opinions. Columnists in the Union-Tribune Community Press are identified clearly to set them apart from news reporters.