Downing Street has so far rebuffed calls for major changes to its Brexit blueprint and ruled out the prospect of cross-party talks ahead of the start of official negotiations with the European Union next week.
Foster's rivals in Northern Ireland, such as Sinn Fein's Gerry Adams, have objected, describing any partnership between the Conservatives and the DUP as "a coalition of chaos".
The Prime Minister held talks with Arlene Foster, leader of the DUP on Tuesday, with a view to a "confidence and supply" agreement, in which the smaller party would support the Conservatives on budgetary and confidence votes.
A failure to gain support from the Northern Irish party would risk the Queen's Speech being voted down next week, and Jeremy Corbyn has said Labour will be pushing hard for that outcome. The comments from the two former premiers will embolden those Tory ministers who are privately plotting to soften May's stance on Brexit, potentially even allowing the United Kingdom to stay in the single market and customs union.
In a TV interview, Mr Halfon added: "I genuinely believe we face potential calamity as a party because people do not understand our values, they see us just in terms of austerity".
The EU has said that sufficient progress must be made on these issues before trade deals can be discussed, though Britain had argued the talks should take place simultaneously.
But Ruth Davidson, the pro-EU leader of the Conservatives in Scotland, called on May to "reopen" the government's Brexit plans.
As May attempts to cobble together a majority, the EU's chief Brexit negotiator Michel Barnier warned that time was passing in an interview on Tuesday.
The European Union wants clarity from London as soon as possible on whether it intends to stick to its stance towards Brexit negotiations or alter it following last week's election, said Guy Verhofstadt, the European Parliament's point person for Brexit.
And speaking in Paris, alongside French president Emmanuel Macron, she claimed: "The timetable for the Brexit negotiations remains on course and will begin next week".
The Times newspaper said finance minister Philip Hammond would push May not to leave the customs union - an arrangement which guarantees tariff-free trade within the bloc but prohibits members from striking third-party trade deals.
May tried to reassert her shattered authority at the weekend by announcing her new cabinet - with no changes among her top team. Gove, a long-time opponent who was dismissed when May became prime minister previous year, will now serve as environment secretary.
May has said that the party, with its 10 crucial MPs to make up a majority in the House of Commons for the Conservatives, will have no veto on key policies.
DUP leader Arlene Foster said there had been "positive engagement" so far.
"The Labour Party stands ready to offer strong and stable leadership in the national interest", he said.
Owen Smith, who challenged Mr Corbyn for the leadership previous year, is predicted to become Shadow Northern Ireland Secretary.
One possible effect of Brexit is the "hardening" of the border between UK-controlled Northern Ireland and the state of Ireland.