|3-17-15 Test Security and Social Media Conflict - Important Issues and Questions|
Star Ledger - Search for PARCC leaks raises new questions about test security measures
Students take a practice version of the new PARCC tests. (Ty Wright | AP Photo)
When New Jersey switched to computerized standardized tests this school year it eliminated concerns about the security of test booklets and whether an erasure mark signified an honest mistake or an attempt to cheat.
But with new tests come new test security issues -- and new controversies.
Because New Jersey adopted the Partnership for Assessment of Readiness for College and Careers (PARCC) exams, its students are answering some of the same test questions as some students in 10 other states, and even students in the same state are taking the tests at a different time.
With a national testing window that began in mid-February ending in early April, PARCC questions could hit the internet weeks before a student's test, jeopardizing the integrity of the exam, said Gregory Cizek, a professor of educational measurement and evaluation at the University of North Carolina.
In response, New Jersey has continued many of the same security measures it implemented in the past, including an online monitoring program designed to spot any test questions or references to specific reading passages posted online, according to the state Department of Education.
Though not new or uncommon, according to testing experts, the online patrolling program has sparked more debate over the already polarizing PARCC tests.
The superintendent of Watchung Hills Regional High School called the online monitoring "disturbing" in a letter to other superintendents that was leaked online and went viral on social media. Some likened the program to spying and the national Parent Coalition for Student Privacy said it's a cover-up to protect a "a very, very shoddy product."
"We will never know how bad (PARCC) is unless students talk about it and teachers talk about it," said Leonie Haimson, the coalition's executive director.
Pearson, the international testing company that provides the online platform for PARCC, said in a statement it stands by the security program run by Caveon Test Security, a subcontractor based in Utah. PARCC and the New Jersey Department of Education defended the measure.
"When students circulate test questions online, it's akin to handing out test questions on the steps of the school -- except in this case it can be seen globally," Department of Education spokesman Michael Yaple said.
Protecting the test
Founded by testing industry veterans in 2003, Caveon has led cheating investigations in Atlanta and the District of Columbia and provided test security for graduate school admissions exams and licensure and certification assessments, Vice President Steve Addicott said.
The company offers clients five major services: consulting, data forensics, security investigations, test development and web patrol.
"We are a company that believes in quality testing and the importance of trustworthy test results," Addicott said.
Until recently, most of Caveon's web patrol clients were companies trying to protect licensure and certification test questions from organizations that "brazenly steal to package and resell," Addicott said.
But as standardized testing has become more high-stakes, the company has picked up contracts with a handful of states the past three years, Addicott said.
Searching publicly available websites and social media channels, Caveon "continually patrols the internet looking for inappropriate sharing or discussion" of its clients' intellectual property, Addicott said. New Jersey will pay $96,574 for the service, according to its PARCC contract.
"We work very closely with our client to gain a sense of the test's nomenclature so we might get snippets of items, descriptions of the test," Addicott said. "We work with them to kind of get as much understanding as we can of how people might talk about the tests, and then we use a bunch of different search technology to cast a really broad and deep net, and then our team, our analysts, spend lots of hours culling through the noise to hone in on what really appears to be a threat."
If a student shares a photo of a test question or recalls a question from memory, it's a breach of test security, Addicott said. But it's not always clear whether online discussion about a test is a breach, so Caveon alerts clients to both obvious and possible threats, he said.
The process is similar to what the SAT and ACT creators do in the week before the test, when booklets have been shipped to testing sites and could be compromised, said Wayne Camara, senior vice president of research for ACT.
"It's not uncommon for folks to do this kind of thing," Camara said. "But I would say for a state test, and a PARCC test, it is actually more critical that they do it because they are leaving the tests out there for so long."
The PARCC tests, administered in math and English to students in grades 3-11, come in six to eight different versions of each test. So while there is no guarantee a question leaked online will appear on another student's test, at least some students would see it on the exam if they are in the same grade, PARCC spokesman David Connerty-Marin said.
Cizek, one of several testing experts brought to New Jersey a few times a year to provide guidance to state officials, said it's critical test questions or references to reading passages aren't posted online during testing windows.
"We want to make sure the test scores really mean what they are supposed to mean," he said. "If you have a test score that is artificially inflated because the student has prior access to test materials, that not only invalidates the score but it's unfair to other students who didn't have that knowledge."
In New Jersey, when the Department of Education learns of a test breach, it immediately alerts the district, which has authority over any potential discipline, Yaple said. Neither Caveon, Pearson nor the PARCC consortium is involved in the disciplinary process, Connerty-Marin said.
State officials would not say how often Caveon has flagged potential breaches.
Protecting the students
After the Department of Education alerted Elizabeth Jewett of a possible breach, the Watchung Hills Regional High School superintendent wrote fellow superintendents.
In the message, which was leaked online last week and spread on social media, she said the Department of Education told her Pearson initiated a Priority 1 Alert for a test question breach within her school.
Though she initially was told the breach occurred during testing and the student had posted a picture of a question on Twitter, a district investigation determined the tweet was sent after school and did not include a photo, Jewett wrote.
The student deleted the tweet, but the student's parent was unhappy the teen's social media account was monitored, the superintendent wrote.
"I have to say that I find that a bit disturbing -- and if our parents were concerned before about a conspiracy with all of the student data, I am sure I will be receiving more letters of refusal once this gets out (not to mention the fact that the DOE wanted us to also issue discipline to the student)," Jewett wrote.
After the email was published online, Jewett confirmed she sent it and stood by her views.
The Watchung incident led to a social media firestorm with hashtag's like #monitoredbypearson. The security measure even drew a response from a state lawmaker, Patrick Diegnan Jr. (D-Middlesex), who invited Pearson to explain the practice at Thursday's Assembly Education Committee meeting, which Diegnan chairs.
In a statement, Pearson said it welcomes debate, but it's obligated to alert PARCC and state customers to potential breaches.
Students have no expectation of privacy if they leak test questions online, PARCC and Caveon said.
"It seems to me that if someone is posting something on Twitter or on a publicly viewable Facebook page, that if someone reads that, how is that spying?" Addicott said. "It seems like by definition they want the information to be public."
Students need to be mindful that social media is designed for sharing, not privacy, Connerty-Marin said. "When you press the button, what you have put out there will be visible worldwide within seconds," he said. "That's what social media is."
Caveon will not report students who make disparaging remarks about PARCC online and is focused solely on test security issues, Connerty-Marin said. The company would not say whether its search tools can access posts made to social media accounts set as private.
If anyone believes Caveon has overstepped in finding a security breach, they should notify the Department of Education, Yaple said.
Because of the length of time between a security breach being identified by Caveon and a district asking a student to delete the post from the internet, the monitoring program may not be as effective as desired, Cizek said.
However, now that students are aware of it, the online patrolling could be a deterrent for sharing information about PARCC, he said.
NJ Spotlight - Surveillance of Twitter Feeds Turn Attention to Online Test Security
John Mooney | March 16, 2015
Pearson reports that student tweets raise concerns about possible PARCC cheating
Security on standardized tests has always been a thorny issue, from the years of shrink-wrapped test booklets to more recent shoe-leather investigations of educators conspiring to doctor answers.
Now, with the advent of new online testing in New Jersey and elsewhere and the ubiquity of the Internet in general, the ways to check for possible cheating are expanding – and so is debate over those methods.
The latest controversy: News that Pearson, the London-based testing vendor hired this year by PARCC, has been monitoring Twitter traffic – and found evidence that some students may have tweeted messages divulging PARCC questions, or at least parts of them.
Officials in at least two New Jersey school districts – and probably more – said they had been informed by the state about suspicious student messages found on the social media platform. Details of the messages were not disclosed.
Pearson informed the state Department of Education, which then informed the districts through a scripted process of “security alerts” and “corrective actions.”
With the PARCC testing already a lightning rod for criticism in New Jersey, news of the Twitter surveillance -- first disclosed by numerous bloggers -- touched off a social media storm of its own over the weekend.
'Spying' by 'Big Brother' alleged
Some accused Pearson and an accommodating Christie administration of “spying,” while others likened it to Big Brother. The incident even garnered its own Twitter hashtags -- #pearsoniswatching and #peepingpearson.
Former Republican gubernatorial candidate Steve Lonegan tweeted: “How many think this is just WRONG?”
Such monitoring of the Internet is not new when it comes to modern-day standardized testing security. Recent cases in New York City and California saw students caught and punished for messaging or publicly posting cell phone pictures of test questions.
The College Board, which administers the college entrance SAT and Advanced Placement tests, explicitly warns against that practice and prohibits students from bringing cell phones into the testing. The worry over divulged questions is one of the reasons its tests are all given at the same time, the College Board said.
That’s not the case with the PARCC exams, which are administered over a month-long period, on different dates in different districts.
The PARCC testing has its own set of security rules, including a required pledge by test administrators and proctors not to discuss the test content with anyone, including on social media. It is not that explicit with students, but does prohibit cell phones or any Internet-accessible devices in testing sites.
Still, reports about Twitter feed monitoring especially touched a nerve at a time when student data privacy has become such a sensitive point.
A bill is already pending in the Legislature to toughen privacy protections pertaining to the testing. Another bill would set a statewide policy for the growing number of families who are refusing to have their children take the PARCC tests.
Just last Thursday, state Education Commissioner David Hespe came before a Senate committee to defend the tests, although the issue of test security never came up.
That prompted one critic to question whether the department was telling all it knows.
“Student data privacy was raised at Thursday's hearing and, like nearly every other concern that was raised that day, was dismissed by the Commissioner as misinformation,” said Susan Cauldwell, a leader of Save Our Schools NJ, a group critical of the new testing.
“In the eyes of many NJ parents, the credibility of the DOE is suspect,” she said.
Both Pearson and PARCC over the weekend defended the use of the monitoring.
“We welcome debate and a variety of opinions,” wrote Stacy Skelly, a Pearson spokeswoman. “But when test questions or elements are posted publicly to the Internet, we are obligated to alert PARCC states. Any contact with students or decisions about student discipline are handled at the local level.”
A spokesman for the state Department of Education wrote a lengthy response defending the practice as well.
“Each year, we see test breaches where students use cellphone cameras to post test questions publicly online, or they post a description of the content of a test question publicly online, so anyone with an Internet connection can see,” Yaple wrote.
“Test breaches have occurred every year, even with the old paper tests. Likewise, test security measures are not new, nor are they unique to this test.”
Yaple yesterday could not provide the number of such cases this year or in prior years.
He promised his department would review any allegations of student privacy being violated.
“The Department always wants to be vigilant to ensure vendors are acting appropriately,” he wrote. “The concern with test breaches is when questions are posted publicly, for any person with an Internet connection to see, and it is our intent to ensure that no one intrudes upon any student’s personal space.
“If any parent or educator believes the company has over-stepped its bounds, let the Department know and it will look into the matter.“
The districts involved weren’t much talking publicly themselves, at least not intentionally. The stir began when an internal email from Elizabeth Jewett, superintendent of Watchung Hills Regional High School District in Somerset County, was leaked to Bob Braun, a prominent education blogger and former Star-Ledger columnist.
In the email, which she sent to several colleagues in other districts, Jewett said her district was called at 10 p.m. on March 10 by a state official who reported the potential security breach discovered on Twitter.
Initially, according to Jewett’s email, the state said a student had tweeted a picture of the test while he was taking it, but then followed up and confirmed that there was no picture and the information was apparently tweeted after the student had left school.
She said in the email that the state official had told the district that the student should be disciplined, but Jewett did not disclose what, if any, discipline was taken.
She acknowledged in the email that she was taken aback by the news that Pearson was monitoring Twitter traffic.
“I have to say, I find that a bit disturbing,” Jewett wrote. “If our parents were concerned before about a conspiracy with all the student data, I am sure I will receive more letters of refusal once this gets out (not to mention that the DOE also wanted us to issue discipline to the student).”
Jewett would not comment further this weekend, but she posted a letter to the Watchung Hills community on the district’s website acknowledging the email – while stressing she had nothing to do with its release -- and standing by her comments. She subsequently confirmed that two additional students had been cited in a separate incident the next day.
“I completely stand behind my comments as they represent not only my views and concerns; they also represent the views and concerns of our Board of Education,” she wrote in the community letter.
“Our main concern is, and will always remain, supporting the educational, social and emotional needs of our students,” she concluded. “The privacy and security of student information remains the utmost priority for our district.”
Yaple, the department spokesman, did not directly address whether the department had told the district to discipline the student, but said that would not be appropriate.
“The local school district has authority over discipline,” he said. “The DOE does not discipline students.”
Not all superintendents were surprised by the monitoring, saying that it is a byproduct of a world where even innocuous tweets or other messages are public.
“I can understand people not being happy that a testing company is monitoring their children on social media,” said Erik Gundersen, superintendent of Pascack Valley Regional High School District. “However, I'm not sure why people are surprised that Pearson may be monitoring social media for test security breach information.
“My understanding is that most large companies constantly track what people are saying about their products and what is trending on social media. We strive to make sure our students know that when the post something online, it is public for all the world to see.”
Garden State Coalition of Schools