Embattled Virginia Gov. Ralph Northam huddled with his Cabinet on Monday as pressure continued to build for him to resign over racist photos in his 1984 medical school yearbook.

Members of the state General Assembly returned to Richmond on Monday, but House Speaker Kirk Cox said lawmakers were focused on working on legislation and cautioned “hesitation” before moving to toss Northam from office.

“I think there’s a rightful hesitation about removal from office because obviously you have to consider that to some degree you’re overturning an election,” Cox, a Republican, told reporters.

Still, Cox said “regardless of the veracity of the photograph” he believes Northam, a Democrat, has lost the confidence of Virginians and cannot continue to govern.

Calls for Northam to step down have come from Democratic and Republican lawmakers since the photo showing a person in blackface and another in a Ku Klux Klan robe on his medical school yearbook page surfaced last Friday.

Northam, who at first apologized for the photo and then the next day denied he was in the picture, also met with senior staffers on Sunday evening, but Lt. Gov. Justin Fairfax was not part of the group.

Fairfax, who would become governor if Northam resigned, threatened to sue a conservative website for publishing a “defamatory and false” story alleging he sexually assaulted a woman at the 2004 Democratic National Convention.

“The person reported to be making this false allegation first approached the Washington Post … more than a year ago, around the time of the Lieutenant Governor’s inauguration,” Fairfax said in a statement issued Sunday evening.

He said the paper “carefully investigated the claim” for several months and after finding “significant red flags and inconsistencies within the allegation” opted not to publish a story.

Big League Politics, which first posted the yearbook photos, reported Sunday that a woman named Vanessa Taylor, a fellow at Stanford University, made the accusations.

The report posted a screenshot of Taylor’s Facebook posting, which didn’t identify Fairfax by name.

But the website linked him to the accusations because Taylor referred to her attacker as a man who won statewide office in 2017 and “seems increasingly likely that he’ll get a “VERY BIG promotion.”

Sens. Mark Warner and Tim Kaine of Virginia, both Democrats, have called for Northam to step down.

Former Va. Gov. Terry McAuliffe said he believed his fellow Democrat will ultimately “do the right thing” and resign.

“Ralph is a good, moral, decent man. And may have made some mistakes in his past. We all have made mistakes. Ralph will do the right thing for the commonwealth of Virginia. He will put Virginia first. And I think that will happen relatively soon,” McAuliffe said on CNN’s “State of the Union” on Sunday.

McAuliffe said Northam’s excuse doesn’t hold water.

“You know if you put black paint on your face,” he said. “You know if you put a hood on. And so if it isn’t you, you come out with immediately and say this is not me.”

After apologizing on Friday, Northam backtracked on Saturday.

“I will not excuse the content of the photo. It was offensive, racist and despicable,” he said. “I believe now that I am not either of the people in that photo.”

During the news conference Northam revealed he once applied dark shoe polish on his face when he posed as Michael Jackson during a dance competition in 1984.

With Post Wires