Intermediaries are obliged to help with loans, and part of your agency fee is the cost of handling loans.
I think there must be a division of responsibilities in your contract that the loan can't be done, such as: the loan can't arrive on time due to Party A's uncooperative reasons, or the loan can't arrive on time due to Party B's own reasons.
In your case, the loan failure is your responsibility, but at the same time, it is reasonable for you to apply for changing the loan bank. The intermediary should not refuse, and the intermediary should try its best to help you complete the loan work.
For this time, you will generally know the specific situation with the seller, and there will be no fine. After all, everyone is not a businessman, and buying and selling has human feelings in it.
It is suggested that the next time you buy a house, you should pay the intermediary fee to the intermediary company after the house is transferred.