President Rodrigo Duterte of the Philippines said he did not like the “creeping influence” of the Catholic Church, a day after the presidential spokesman accused the Church of working with Communist rebels to overthrow the government.

Speaking on Wednesday at the 31st founding anniversary of the Philippines Department of Environment and Natural Resources, Duterte said the Church “sometimes runs counter to what the government believes to be good for the people, at least in this temporal life.”

The president was speaking specifically of the government’s family planning policies, which are strongly opposed by the country’s bishops’ conference.

“Yesterday, I read somebody said that we have to implement the family planning. To be frank, the family planning, with due respect, I don’t want to pick a fight, it has something to do with the objection of religions, not all,” Duterte said.

The Responsible Parenthood and Reproductive Health Act was passed in 2012 and survived a court challenge with most of its provisions intact in 2014.

Duterte has often clashed with the Church in the Philippines, one of only two Catholic-majority countries in Asia.

The bishops have criticized his violent anti-drug program, which has led to the deaths of thousands, and his push to reinstate the death penalty.

Last month, Duterte said God was “stupid” and attacked several of the teachings of the Catholic faith.

Those remarks led the president to arrange a meeting with the president of the Catholic Bishops’ Conference of the Philippines, Davao Archbishop Romulo Valles, which is expected to take place on July 9.

On Tuesday, presidential spokesman Harry Roque – who is a member of the four-member committee arranging the Church-government dialogue – said “it’s not far-fetched to say” the bishops could join forces with the country’s Communist rebels to try and oust Duterte.

“There are elements of the Catholic Church that would never accept the President for who he is,” Roque told reporters. “There will be efforts on the part of CPP-NPA [a communist rebel group] to penetrate all sectors of society. That’s not new.”

Manila Auxiliary Bishop Broderick Pabillo immediately responded, challenging Roque to “name names.”

“He is just spreading rumors. That is not responsible reporting. This is one way of spreading false news,” the bishop said.

“That only shows how insecure they are. They are so blinded by their fears and their bias that they cannot see their mistakes. They deflect on others their inefficiencies,” Pabillo told CBCPNews, the official news service of the bishops’ conference.

Bishop Rey Evangelista of Imus, the head of the conference’s public affairs body, also denied any effort to undermine the president.

“It is just a fabrication that came out of somewhere. It cannot come from the Church. I can assure you of that,” he said.