Sen. John McCain (R-Ariz.) hit back Wednesday at President Barack Obama’s criticism of him and his Republican colleagues, declaring that Obama “must be held responsible” for the Sept. 11 terror attacks in Benghazi, Libya.

“I understand today that the president of the United States took some umbrage at statements that Senator Ayotte, Senator Graham and I made concerning this issue,” McCain said on the Senate floor.

The three said earlier Wednesday they were calling for a special committee to investigate what happened in Libya and ripped U.N. Ambassador Susan Rice as an unacceptable possible replacement for Secretary of State Hillary Clinton over her initial comments calling the attacks a response to an anti-Islam video. Obama told reporters during a news conference Wednesday that “when they go after the U.N. ambassador…then they’ve got a problem with me.”

“We believe whoever it is must be held responsible, I say to the president of the United States — most importantly, the president of the United States who is commander in chief, who so far in my view has not exercised those responsibilities and also not informed the American people of the facts,” McCain said. “This president and this administration has either been guilty of colossal incompetence or engaged in a cover-up, neither of which are acceptable to the American people.”