The men - a Briton and an Italian - were killed by their captors. A Nigerian official said the two died in the crossfire during the rescue attempt, Prime Minister David Cameron said.
Chris McManus and Franco Lamolinara were taken captive in May. The pair had been working on a bank construction project in Nigeria, a country that has seen a rise in violence linked to the Islamist extremist group Boko Haram.
Britain's Foreign Office confirmed two men were held by terrorists associated with Boko Haram, and a senior British government official said the kidnappers appeared to be from an al-Qaida-linked cell within Boko Haram, but not within the group's main faction.
The British prime minister said that after months of not knowing where the men were, U.K. authorities had "received credible information about their location." Believing the men's lives were in "imminent and growing danger," a rescue operation was mounted, Cameron said.
"The terrorists holding the two hostages made very clear threats to take their lives, including in a video that was posted on the Internet," Cameron said.
U.K. officials had asked journalists not to publicize the video, in which the kidnappers claimed they belonged to al-Qaida and threatened to kill McManus and Lamolinara if their demands were not met.
Late Thursday, the exact circumstances of how and when the men died remained unclear.
Cameron said "the early indications are clear that both men were murdered by their captors, before they could be rescued." But a senior security official in Nigeria said the two hostages apparently died in the crossfire. He requested anonymity because he was not authorized to speak to journalists.
The Nigerian security official said the failed rescue attempt took place in Sokoto, a quiet city in Nigeria's northwest. An AP reporter there said the military surrounded a house and gunfire was heard for much of the afternoon and into Thursday night.
The reporter saw an ambulance ferrying away wounded people, but could not get close enough to see who was inside. Security forces cordoned off the area, blocking journalists from getting within a kilometer (around a half mile) of the site.
Nigerian President Goodluck Jonathan condemned the killing of the hostages. He said in a statement "the perpetrators of the murderous act, who have all been arrested, would be made to face the full wrath of the law."
It was unclear what - and how many - arrests he was referring to. Nigeria's secret police spokeswoman Marilyn Ogar declined to comment, and the army and police could not immediately be reached for comment.
A number of foreigners have been kidnapped while working in Nigeria in recent years - including many who worked in the oil industry. But the kidnapping of McManus and Lamolinara in northwest Nigeria was highly unusual for that region of the country, an arid land that approaches the Sahara Desert and Nigeria's border with neighboring Niger.
Boko Haram, which means "Western education is sacrilege" in the Hausa language of Nigeria's Muslim north, has been waging an increasingly bloody fight against the nation's weak central government.
Western diplomats and analysts say Boko Haram has had contacts with two other al-Qaida-influenced terror groups in Africa. However, they say the sect has splintered, with one wing now willing to use extreme violence. The sect also has begun targeting schools and Christians in Nigeria, a multiethnic nation of more than 160 million people.
A senior British government official said Thursday the kidnappers appeared to be from an al-Qaida linked cell within Boko Haram, but not within the main Boko Haram faction. The official spoke on condition of anonymity because of the sensitivity of his work.
"They appear to have grown in strength," the official said. "Last year, they had diminished in their capability - largely because of the Nigerian authorities, but they have regrouped and it appears they are getting external help."
McManus was working for the construction company B.Stabilini when he was kidnapped May 12 by gunmen who stormed his apartment in Birnin-Kebbi. Lamolinara was also abducted. A German colleague managed to escape by scaling a wall, but a Nigerian engineer was shot and wounded.
In a statement, McManus' family said they were devastated by the news of his death. "We are also aware of the many people who were working to try and have Chris returned to our family, and his girlfriend," the statement said. "We would like to thank all of them for their efforts.
"We knew Chris was in an extremely dangerous situation. However, we knew that everything that could be done was being done. "
Italian Premier Mario Monti said Cameron had conveyed the news to him by telephone. Monti said in a statement that Nigerian and British authorities had determined the operation was the "last window of opportunity to save the hostages' lives."
The statement said the situation on the ground had accelerated recently, posing an "imminent danger to the lives of the hostages." The Italians were only informed once the operation was under way, according to the statement.
Monti asked Nigeria's president to provide him with a "detailed reconstruction" of what went wrong during the operation as soon as possible, a statement from Monti's office said. The two spoke by telephone Thursday.
The mayor of the tiny northern Italian town of Gattinara, near Turin, where Lamolinara lived with his wife and two teenage children, said he would proclaim a day of mourning.
The town's mayor, Daniele Baglione, told the ANSA news agency the community was in shock, convinced that Lamolinara would have been rescued successfully. For nearly a year residents adhered fully to the directives of the Foreign Ministry to not utter a word about the kidnapping, he said.
"We were so hopeful," he told ANSA. "We didn't expect it. The Foreign Ministry asked us to keep quiet, and the whole town managed to do so and rally around the family."