WASHINGTON, D.C. — President Joe Biden attended Mass for the first time since taking office, worshipping Sunday at the church he frequented when he was vice president.

Biden, the nation’s second Catholic president, picked Holy Trinity Catholic Church in Washington’s Georgetown neighborhood, a few miles from the White House. It’s where the nation’s only other Catholic president, John F. Kennedy, often went to Mass.

Biden entered through the front entrance, where a Black Lives Matter banner was hanging on one side and a banner with a quote from Pope Francis was on the other: “We cannot tolerate or turn a blind eye to racism and exclusion in any form and yet claim to defend the sacredness of every human life.”

The president, in a brief exchange with reporters, said the service was “lovely.” Biden was accompanied to church by his son, Hunter, and two of his grandchildren, Finnegan and Maisy.

His motorcade made a brief stop on the way back to the White House for carryout from Call Your Mother, a popular deli near the church. The president remained in his armored vehicle, while his son picked up the order.

White House press secretary Jen Psaki told reporters on Friday that Biden had not yet settled on a home church in the nation’s capital, but said that she expected Biden will continue to regularly attend services during his presidency.

At home in Delaware, Biden and his wife, Jill, were regulars at St. Joseph on the Brandywine in Greenville. They alternated between the Saturday and Sunday services depending on their travel schedules throughout the 2020 campaign. Catholic faithful have an obligation to attend Sunday services, but church teaching allows for the commitment to be fulfilled by attending a service on the evening of the preceding day.

The newly-sworn in Democrat certainly has plenty of parish choices in Washington: Four Catholic churches sit within 2 miles of the White House; Holy Trinity is a bit farther.

On the morning of his inauguration Wednesday, Biden and his family, along with Democratic and Republican leaders of Congress, attended a service at one of those churches, the Cathedral of St. Matthew the Apostle. The church hosted Kennedy’s funeral service in 1963.

With the coronavirus still surging in the capital city, Biden is bound to see small crowds wherever he goes. For the time being, rules in the District of Columbia limit gatherings at houses of worship to 25 percent of capacity or 250 people, whichever is less.

Previous presidents have made a wide variety of worship choices — or none. Not far from the White House is New York Avenue Presbyterian, which maintains the pew where Abraham Lincoln once worshipped. Even closer is St. John’s Episcopal Church, walkable across Lafayette Square from the White House for the presidents who have made a historic practice of worshipping there at least once.

St. John’s was thrust into the headlines this summer when police forcibly dispersed protesters so President Donald Trump could pose with a Bible outside its butter-yellow front doors. But its status as the “Church of Presidents” dates to James Madison, and it’s accustomed to the special scrutiny that comes with hosting commanders in chief. Trump, who frequently spent Sundays at his namesake golf club in northern Virginia, was not a regular churchgoer.

President Bill Clinton and his wife, Hillary, became members of Foundry United Methodist Church, a short drive from the White House that also counted the 19th president, Rutherford. B. Hayes, as a member.

President Jimmy Carter, who in post presidency life taught Sunday school, worshipped dozens of times at Washington’s First Baptist Church during his time in the White House.

Associated Press writers Will Weissert and Elana Schor contributed to this report.