You can return the house after paying the down payment. You can ask to return a house after paying the down payment, but if you don't want to buy a house for your own reasons, you need to bear the liability for breach of contract.
Article 1 14 of the Contract Law: The parties may agree that one party shall pay a certain amount of liquidated damages to the other party according to the breach of contract, or agree on the calculation method of the amount of damages for breach of contract. If the agreed liquidated damages are lower than the losses caused, the parties may request the people's court or arbitration institution to increase them; If the agreed liquidated damages are excessively higher than the losses caused, the parties may request the people's court or arbitration institution to reduce them appropriately. If the parties concerned pay liquidated damages for delayed performance, the breaching party shall also perform the debt after paying the liquidated damages.