The only things that matter are the rate, the terms, and whether the broker or lender you use can perform when it's time to close. Brokers shop many banks, including the mortgage banks. In the end, if it's a conventional loan, it's going to be sold to Fannie or Freddie. If the bank does the loan, they will probably retain the servicing of the loan. If it's brokered, any random servicer may be the company to whom you send the check every month. And like any other major purchase, you should shop the various money "stores" until you get the best deal.