I posed the same sort of idea back at the beginning when I was just starting to get interested in investing.
It's basically you trying to be logical, but without really understanding the logic behind investing itself. So it's a logic fail.
Advisers aren't in the business to make you money. They are in the business to make THEMSELVES money, and then their company money. You are just the vehicle they are using to achieve these goals. If they happen to make you some money using your money, that's a happy accident. Advisers use all kinds of spiels to draw in clients: they'll tell you they are looking out for you, you're too busy/naive, the stock market is super complex and scary, you aren't nearly as smart as them, that they have your best interests in mind, can beat out any other group or funds or whatever... they will tell you
anything to get your business, and they are very well trained to manipulate the facts/data/perception and prey on those that are unable or unwilling to do the research/learning themselves. Unless they are fiduciaries (and no, most all of them are not) then they are not even liable to invest your money in ways that protect and serve your best interests.
Have you read the stock series by Jim Collins? He wrote a book that is a fun read based off the website.
http://jlcollinsnh.com/stock-series/I'll direct you to one of the chapters that is quite relevant:
http://jlcollinsnh.com/2012/06/06/why-i-dont-like-investment-advisors/