The vote comes amid the country's deep economic crisis — including a 24 percent unemployment rate — and huge public discontent with plummeting living standards.
Nikolic, a somber former cemetery manager, said Sunday he's certain of a victory.
"It's not the first time. But this time it's definite," Nikolic, 60, said after voting. "Serbia is anxiously awaiting changes, the changes that are necessary. It cannot go on like this any longer. I think that either tonight, or in two weeks, we can openly discuss how to move Serbia forward."
Incumbent President Tadic said if he and his Democrats win, they will quickly form a new government.
"I expect that Serbia will continue on its reform path," Tadic, a charismatic 54-year-old former psychology professor, said after casting his ballot. "Better life, better living standards for ordinary people is our strategic goal."
Election monitors said turnout several hours before the polls were to close was about 32 percent, the largest since 2000 when pro-democracy forces ousted Milosevic from power. The turnout appeared to indicate that Serbs regard the election as crucial in shaping the future of their country.
One voter — Ljubinka Marjanovic, a high school teacher from Belgrade — said: "It's not much of a choice we're having: Tadic's corrupt government or those ... nationalists who want to return us to the past. But I had to vote for Europe, for the future of my grandchildren."
Nikolic claims to have shifted from being staunchly anti-Western to pro-EU. But that is not taken at face value by many Serbs and Western officials because the former far-right politician only a few years ago stated that he would rather see Serbia become a Russian province than an EU member.
Tadic — considered a moderate — advocates quick EU entry, while Nikolic — who has Russian support — says he wants to see Serbia "both in the West and East."
Nikolic says Serbia should not be an EU member, if the bloc demands that Serbia give up its claim on Kosovo, which is considered the cradle of the Serbian state and religion.