A recent poll finding most Americans don’t care about this year’s World Cup says more about polling challenges than it does about interest in the global soccer tournament.
An Ipsos/Reuters poll this month said 66 percent of Americans don’t plan to follow the World Cup this June and July in Brazil, and just 7 percent expect to follow it closely. But that’s less worrying than it sounds for tournament organizer FIFA, or for ABC, ESPN and Spanish-language network Univision.1
That’s because the poll was conducted only in English, a significant shortcoming. You can’t get complete data on Americans’ interest in the World Cup unless you talk to people who speak languages other than English.
“If you don’t get the Spanish-dominant population, you’re not going to pick up the level of Latino interest in the World Cup,” Rodolfo de la Garza, a professor of international and public affairs at Columbia University, said in a phone interview. De la Garza has studied polling among Latinos, was born in the U.S. and greatly prefers football to fútbol. “The group that’s interested is the immigrants. They organize around soccer,” he said. Among those Latinos who responded to the Ipsos poll in English, 16 percent said they plan to follow the World Cup closely — or more than three times the proportion of the rest of respondents.
Chris Jackson, research director at Ipsos Public Affairs, said the firm’s surveys for Reuters are weighted by race and ethnicity but not by country of origin or first language. “We do capture some information on Hispanic and Asian country of origin. Unfortunately we do not capture information on language spoken at home,” he said in an email. He added, “We do not have prior data on the World Cup. Our partnership with Reuters started just after the last Cup and we had not yet developed that part of our materials.”
Leaving Spanish-speakers — and others who prefer languages besides English — out of the conversation is becoming increasingly untenable for pollsters, and not just when it comes to soccer. The number of Americans age 5 and older who spoke a language other than English at home increased by 158 percent from 1980 to 2010, to 59.5 million people, or 20.6 percent of the population. For 62 percent of those nearly 60 million Americans, their language was Spanish or Spanish Creole.
Excluding people who’d only respond in Spanish doesn’t matter much when pollsters are measuring characteristics for which language doesn’t matter. If Latinos who respond in English respond just as those who don’t, then their answers suffice. That’s true even if Latinos are underrepresented in the survey, since pollsters typically weight responses of underrepresented groups more heavily.
But unfortunately for pollsters, Spanish-dominant Latinos often respond differently from English-dominant Latinos. David Dutwin, executive vice president and chief methodologist at the survey-research firm SSRS, explained in an email why his pollsters almost always conduct surveys in Spanish: “Hispanics interviewed in Spanish are generally half as likely to own a home, half as likely to be single, nearly half as likely to be employed full time; 1.5x more likely to be a parent (and they are older); four times more likely to have never graduated high school (near 50 percent!), slightly more independent and slightly less Democratic (independent here almost certainly meaning, unaffiliated and nonpolitical); half as likely to be registered to vote; and 1.5x more Catholic; than Hispanics whose surveys are done in English.”
The Pew Research Center, in its Hispanic Trends Project’s 2012 National Survey of Latinos, identified other crucial differences between the two groups by their language of interview. Just 6 percent of Spanish-language interviewees were born in the U.S., compared to 76 percent of Latinos who took the survey in English. Just 5 percent of the first group were Puerto Rican; 13 percent of the second were.2
Language preference among Latinos in the U.S., or any group, exists on a spectrum. Researchers sometimes separate Latinos into three language groups: English-dominant, Spanish-dominant and bilingual. The bilingual group might choose to answer surveys in either English or Spanish.
In an ongoing, unpublished study using Pew survey results, Columbia’s de la Garza and Alan Yang find that bilingual Latinos are the most likely of the three groups to be registered and to vote.3
FiveThirtyEight editor-in-chief Nate Silver wrote about how polls answered by too few Latinos who primarily speak Spanish may have contributed to underestimates of Democratic support in the 2010 and 2012 national elections. Immigration is another issue for which pollsters might misgauge public opinion if they don’t survey in Spanish, survey researchers say.
Latinos aren’t the only Americans whom pollsters might miss by surveying only in English. Last year Pew reported the proportion of U.S. Asian adults who said they spoke English less than “very well.”4 At least 45 percent of many ethnic groups — including Chinese Americans and Korean Americans — said that about their English skills, compared to 9.5 percent of all American adults.
More pollsters are recognizing the value of surveying in other languages, particularly Spanish, the country’s dominant second language. Langer Research Associates has conducted ABC News/Washington Post polls in Spanish as well as English since last year, and the Bloomberg Consumer Comfort Index in both languages since 2012. Pew has used Spanish in some polls since 2003, and has regularly used it in national polls since 2011.5
The trend continues even though it adds cost and complexity to a survey project, said Evans Witt, principal and chief executive of Princeton Survey Research Associates International. “For example, the PSRAI Omnibus surveys — a weekly, quick-turnaround, multi-client survey of 1,000 respondents on cell phone and landline — added Spanish-language interviewing for 2014 after clients’ requests.”
The American Association for Public Opinion Research, pollsters’ professional association, has no standards on including languages other than English, according to AAPOR president Rob Santos. Some pollsters decide on a case-by-case basis about Spanish-language interviews.
“Some national surveys do and some don’t,” said Cliff Zukin, professor of public policy and political science at Rutgers University. “It really depends on the cost and what the survey sponsor is willing to pay.”
Jay Leve, founder of SurveyUSA, said in an email that his polling firm always has offered bilingual surveys, at added cost. The cost isn’t always worth it: “In round numbers, if we are surveying an area where we know (from U.S. census) that the population is 30 percent Hispanic/Latino, we may get 4 percent to 5 percent who elect to take the survey in Spanish,” he said. The rest of Latino survey participants answer in English.
Leve nonetheless insists on a Spanish option for some polls. When clients who were seeking to poll on immigration refused to pay extra for surveys in Spanish, SurveyUSA has turned down the jobs, Leve said.
“We do offer Spanish-language interviewing, but only when our expectations about the Latino proportion of the population are sufficiently high,” Monmouth University Polling Institute director Patrick Murray said in an email. “When you get down to the level of likely voters, the number of Spanish-only (or non-English proficient speakers) qualified respondents we contact in a sample of 1,400 is something you could count with your fingers. The added cost for translating that questionnaire into Spanish and having our Spanish-speaking interviewers doing heavy re-contacts to track them down doesn’t justify the presumed benefit (i.e. it would affect our total vote estimate by a couple of tenths of a percentage point at most).”
In 2012, Matt Barreto, co-founder of the polling and research firm Latino Decisions, wrote a piece criticizing fellow pollsters, including Monmouth, for missed forecasts of the Latino vote in the presidential race. Murray said the poll in question was an outlier, and other polls before and after showed more-accurate levels of support for President Obama among Latinos. “Just because this happens from time to time, doesn’t mean I am happy when it does,” Murray said. “But … that outlier would not have been ‘fixed’ if we had used Spanish-language interviewing in that poll.”
Soccer polls might need Spanish interviews even more than political polls do.
“Among non-English speakers, interest in the World Cup is probably 10 to 20 times higher than among English speakers in the U.S.,” Barreto said in an email. “This is not just among Spanish speakers, but pretty much among any immigrant population in the U.S.” For them, Barreto said, “Soccer is the No. 1 sport, and the World Cup is the No. 1 sporting event.”
Barreto’s estimate of 10 to 20 times is just that, but there’s plenty of data showing that soccer is far more popular in other countries than in the U.S., and therefore probably more popular among non-English-speakers in the U.S. Nielsen’s sports-media review last year showed that the NFL dominates U.S. sports viewership, but in many countries around the world — including Italy, South Africa and Indonesia — people spend more collective time watching soccer than watching all other sports combined.
Another indicator of the importance of language in U.S. sports surveys comes from ESPN’s annual sports poll. The percentage of respondents who said they were fans of pro soccer, international soccer and Major League Soccer was basically unchanged from 2007 to 2013, with one big exception: a jump in each of about 5 percentage points in 2011.
That was the year the poll first added Spanish-language interviews, said Juan D. Rivera of ESPN’s consumer insights group.