ANNAPOLIS, Maryland  — A statue of the U.S. Supreme Court justice who wrote the 1857 Dred Scott decision that upheld slavery and denied citizenship to African Americans was removed from the grounds of the Maryland State House early Friday.

The statue of Roger B. Taney was lifted away by a crane at about 2 a.m. It was lowered into a truck and driven away to storage.

Although most known for authoring the 7-2 Dred Scott decision, Taney was also an important figure in U.S. Catholic history.

He became the first Catholic to have served in a U.S. president’s cabinet when he was appointed Attorney General by Andrew Jackson in 1831. Taney later served as Secretary of War and Secretary of the Treasury under Jackson. He became the first Catholic Supreme Court Justice in 1836, after Jackson nominated him as Chief Justice to replace John Marshall the preceding December.

“Taney was regarded as a very good Catholic,” Father Michael Roach, a Church historian, told the Catholic Review in 2012. “It’s pretty impressive that he made it to the Supreme Court.”

Taney had once served as Attorney General of Maryland, and is still the only Maryland native to have served as Chief Justice of the Supreme Court.

The bronze statue of him was erected in 1872, just outside the original front door of the State House.

Three of the four voting members of the State House Trust voted by email Wednesday to move the statue. House Speaker Michael Busch, a Democrat who was one of the three who voted to remove it, wrote this week that the statue “doesn’t belong” on the grounds.

His comments came after the violent protests in Charlottesville, Virginia, last weekend, with clashes between white nationalists and counter-protesters. A woman was killed when a car plowed into a crowd of people who were there to condemn the white nationalists, who had rallied against Charlottesville officials’ decision to remove a monument to Confederate Gen. Robert E. Lee.

Maryland Gov. Larry Hogan, a Republican, said this week that removing the statue of Taney in Annapolis was “the right thing to do.” Republican Lt. Gov. Boyd Rutherford voted on behalf of the administration to remove the statue.

One member of the trust, Senate President Thomas V. Mike Miller, criticized holding the vote without a public meeting.

“This was certainly a matter of such consequence that the transparency of a public meeting and public conversation should have occurred,” Miller, a Democrat, wrote in a letter Thursday to Hogan.

Miller also noted that Taney was personally against slavery – calling it “a blot on our national character” – and had freed his own slaves early in his life. Taney also was pro-Unionist, and continued to serve on the Supreme Court during the Civil War.

“Taney was not a Confederate officer, and he remained loyal to the Union until his death in 1864,” he said. Maryland was a “border state” during the Civil War, meaning slavery was legal, but the state did not join the Confederacy.

“But regardless of one’s position on this issue, it is insulting to our citizens for the State House Trust to vote on such a matter outside of the public eye,” Miller wrote.

While the statue’s removal was not publicized, a couple dozen onlookers watched as workers started the removal process shortly after midnight Thursday. Some witnesses cheered as the statue was lifted from its pedestal.

The statue was removed two days after Baltimore Mayor Catherine Pugh ordered the removal of four monuments from her city under the cover of night, including another statue of Taney.

Taney was born in Maryland and practiced law in Frederick before becoming the nation’s fifth chief justice. Dred Scott and his wife, Harriet, were slaves who sued for their freedom after they were taken from the slave state of Missouri into territory where slavery had been prohibited by the Missouri Compromise.

This year marked the 160th anniversary of the 1857 decision. In March, a family member of Taney’s apologized to the Scott family in front of the statue that was removed Friday. Charles Taney IV of Greenwich, Connecticut, apologized to the Scotts and all African Americans for the “terrible injustice of the Dred Scott decision.”

Lynne Jackson, a great-great-granddaughter of Dred Scott, accepted the apology for her family and “all African Americans who have the love of God in their heart, so that healing can begin.”

When Taney died, the New York Times wrote, “Had it not been for his unfortunate Dred Scott decision, all would admit that he had, through all those years, nobly sustained his high office.”

Crux Staff contributed to this report.