New Delhi: I have been a psephologist for about 16 years and have had a fair amount of success in predicting seats in an upcoming election. Here is a post-mortem of what I said and what happened in the current round of elections.
Since 2005, I have been working with CNN-IBN and CSDS (Center for studies in developing societies). CNN-IBN engages CSDS for the survey, who do a great job following text book methodology and I use the vote share data from the survey to come out with seat projections which CNN-IBN carries on air.
CNN-IBN prediction: In Madhya Pradesh, BJP will win 136-146 seats, Congress 67-77 and Others 13-21, In Rajasthan, BJP will win 126-136 seats, Congress 49-57 and Others 12-20.
In Chhattisgarh, BJP will win 45-55 seats, Congress 32-40 and Others 7-13. In Delhi, BJP will win 32-42 seats, Congress 9-17, AAP 13-21 and Others 1-5.
According to the Assembly election results: In Madhya Pradesh BJP won 165 seats, Congress 58 and Others 7. In Rajasthan, BJP won 162 seats, Congress 21 and Others 16. In Chhattisgarh, BJP won 49 seats, Congress 39 and Others 2. In Delhi, BJP won 31 seats, Congress 8, AAP 28 and Others 3.
These results show the power as well as limitations of opinion poll based projections. If one simply counts the number of cases out of 13 that the actual results are within the interval projected- then the score is just 4. However, one should see these as having correctly predicted clear and decisive victories for BJP in Rajasthan and Madhya Pradesh. For Chhatisgarh of course we predicted correctly that BJP will win, with much smaller gap than Madhya Pradesh and Rajasthan. So I would count all the three as being good predictions, with Chhatisgarh being very good. As for Delhi, we underestimated AAP support and marginally overestimated BJP but we had the ordering right- namely Congress being in third position with possibility of touching a single digit, BJP being the single largest party. So I would consider this as fair (perhaps a score of 6 on a 10 point scale).
According CSDS vote estimates on air (CNN-IBN), in Madhya Pradesh BJP won 41 per cent votes, Congress 35 per cent and Others 24 per cent. In Rajasthan, BJP 43 per cent votes, Congress 33 per cent and Others 24 per cent. In Chhattisgarh, BJP won 42 per cent votes, Congress 38 per cent and Others 20 per cent. In Delhi, BJP won 33 per cent votes, Congress 23 per cent votes, AAP 27 per cent and Others 17 per cent.
According to Assembly election results: In Madhya Pradesh, BJP won 44.9 per cent votes, Congress 36.4 per cent and Others 18.7 per cent. In Rajasthan, BJP won 45.1 per cent votes, Congress 33 per cent and Others 21.9 per cent. In Chhattisgarh, BJP won 41 per cent votes, Congress 40.3 per cent and Others 18.7 per cent. In Delhi, BJP won 34 per cent votes, Congerss 24.5 per cent, AAP 29.5 per cent and Others 12 per cent.
So one can see that the survey is very good and the errors are generally within the statistically acceptable range. The conversion from votes to seat is a non trivial transformation- since one needs to estimate the distribution of votes across the state in addition to the overall percentage of votes in the state. This requires building an appropriate statistical model. I will write another article explaining my methodology for this step.
One can see that in MP and Rajasthan there was underestimation of BJP votes by a few percentage points. I have analyzed and seen that the error in vote to seat conversion went in the same direction and as a result our prediction was much lower than BJP's actual number of seats. In Chhattisgarh on the other hand, survey estimated the gap between BJP and Congress as 4 per cent while the actual gap turned out to be less than one percent. In this case the error in vote to seat conversion and the error in vote estimate cancelled out and we got a result that was bang on. Of the four I was least confident about Chhattisgarh (I had said so on air) since the estimated gap in vote shares was small.
I would like to add that in my view (and experience) predictive power of any opinion poll done months or few weeks ahead of actual voting is rather poor in India. For one, any such poll can only measure the mood of state or nation at the time of poll and cannot estimate the potential change that can happen close to the voting day. Some psephologists claim to estimate this change by conducting polls at regular intervals and then extrapolate to get an estimate of this change. However, this assumes that the rate of change is constant, again something that is probably not true.
Another reason as to why the pre election opinion poll could be off the mark is that for the final result, only the opinion of people who go and vote matters and we know from experience in India that propensity to vote is not uniform across socio-economic stratums of the society. One can try to factor these in but that can inflate the error.
Exit polls are designed to take care of both these issues. However, choosing respondents in a randomized manner as they exit the booth is rather difficult and our experience has shown that it does not produce a representative sample (as measured by gap between socio-economic profile of the sample and population). Thus we prefer to do a post poll, where in the days following actual voting, we do a proper methodological door to door survey. In the current round, various exit polls had got numbers close to the actual results too for MP, Rajasthan and Chhattisgarh. But for Delhi there was wide variation. And in the past, there have been occasions when exit polls had given an incorrect picture while we got it right with our post poll.
Recently, there had been a big discussion on the proposal to ban opinion polls in the run up to the elections. Here is my take on this issue. I think the polls need to be regulated rather than banned. Indeed, the regulation could be self regulation by the media- such as via Press Council. And the regulation should ensure that each published poll also puts in public domain (say on a web-site) the detailed methodology of sample selection, sample size, socio-economic profile of the sample, the dates when sampling was done, names of the core team members who supervised the survey. Also the methodology used to convert vote estimates to seat conversion should also be in public domain.
We (myself and CSDS) have been very open about this. The sampling methodology is on CSDS website, the sample size is always given on air and the socio economic profile of the sample is also given on the CSDS website ( Lokniti). I have written about the vote to seat conversion in an academic article ( Predicting the 1998 Indian parliamentary election, Karandikar, R. L. ; Payne, C. ; Yadav, Y., ELECTORAL STUDIES; 21, 1; 69-89) and have been talking about it in several seminars.
As has been pointed out, the election commission and the government can perhaps ban telecast of opinion poll findings on TV channels but it probably does not have any control on newspapers. The TV channels had to agree to stiff conditions in order to get uplinking license but newspapers have not agreed to any such conditions. Moreover, an outright ban would just prompt people to write about such survey findings (real or fictional) quoting a reliable source! Even now betting is banned but one does get reports in newspapers about the odds being offered on election outcomes. And then there is the internet. How can the government or the election commission prevent reports about such polls on the social media or blogosphere?
But I do believe that there is a feedback loop- namely if all surveys point in a certain direction, at least some section of voters are likely to be influenced by the same. But this is no reason to ban the opinion poll. After all, newspapers and TV channels also talk about their assessment of the political situation and if they all seem to point in a certain direction, this too has an effect on the electorate.
However, I am for regulation. Indeed, I would go as far as saying that all agencies should be open to an audit by an expert group formed by an autonomous body such as Press Council. As to why I feel that surveys do influence voters - I have indirect evidence. In each of the surveys done by CSDS, one standard question is - whom did the respondent vote for in the last election (last Vidhan Sabha election, if this is a poll for Vidhan Sabha and last Lok Sabha election, if the current one is for the Lok Sabha). Almost invariably the recall for whoever won the last time is much higher than the actual votes, even when the winner from previous poll is set to lose the current election. Thus in 2011, much larger percentage of voters seem to recall having voted for the left front in 2006 though in 2011 they were voting for Trinamool Congress. I may add that our estimate of the Trinamool vote share and seats in 2011 was very accurate. The same was true in Tamilnadu where, while voting out DMK much larger number of respondents seem to recall having voted for them. We have observed this time and again across various states and consistently over the last 15 years that I am involved in psephology. The only explanation that I could come up with is that there is a general tendency to go with the winner. This makes it important that polls be regulated in some manner. Fortunately over the years the buzz that a given opinion poll generates has reduced significantly, with so many agencies doing polls and making contradictory predictions- like for the recent Delhi elections, the range of seats for the Aam Admi Party was 6 to 31(+9) out of 70 seats!
I would also like to add that Opinion polls are the only way that one can find out as to why people voted the way they did. Indeed, an interesting finding of the CSDS poll in MP, Rajasthan and Chhattisgarh is that the gap between BJP and Congress vote shares was higher in rural areas than in the urban areas in each of these states. This is in sharp contrast to the popular view that BJP is an urban party. Indeed, the CSDS website gives breakup of voting intention by various socio-economic groups and makes an interesting read.
(Rajeeva Karandikar is the Director of Chennai Mathematical Institute)
Comments
0 comment