I think many people seriously underestimate how difficult it is to consistently make a good living as a farmer. I don't know of any other occupation or investment where there are so many factors outside your control causing you to lose money. Number one is the weather. It's not something you invest in to get rich, it's something you do because you love it.
Many large and corporate farms that look successful at first glance are often just a write-off for other business interests. And how do the successful family farmers actually build wealth? By owning the land they farm.
And to whomever mentioned contracting out all the work, that's a great option for a small number of acres, but once you get over a certain number (I'd say around 250 acres depending on equipment age), it's far cheaper to own the equipment, do everything yourself, and average the cost over all acres for the lifespan of the equipment. For example, let's say you spend $150k on a new-to-you combine w/corn and bean headers and you only farm 250 acres. If well maintained, I'd say it's reasonable to get at least 20 years of use before replacement, which comes out to $7500/yr and $30/acre over those 20 years. Compare that to paying at least $35/acre for custom harvesting. And of course those are today's rates which will only go up. Sure, you'll spend more than that $150k in the end in maintenance and repairs, but it's still cheaper overall. The more acres you farm, the cheaper that equipment becomes to own.
Now for some real-world numbers on a midwestern dryland farm with relatively poor dirt (in comparison to all the Northern corn belt states). My FIL is a 3rd generation small family farmer and has allowed me to farm a few of his acres... for fun and to try some different things. Since I live in a different state, I pay all inputs and pay him to do everything. Sometimes we'll be back to visit and I might do the work but I'm still using his equipment. This year I planted corn, for which I would normally shoot for a yield of 120-140 bu/acre, but thanks to some untimely dry spells and a poor choice for nitrogen, ended up yielding only 63 bu/acre. I netted roughly $120/acre. BUT, he doesn't charge me rent and ate the cost of nitrogen fertilizer because he felt that decision was his mistake. If you include those expenses, that's another $225/acre and I would've lost $105/acre. Now this was a rough year and having a lot of acres in soybeans helps to offset the losses in corn, but I still suspect it was a break even year overall.
I don't think investing in farmland is a bad decision for diversification, but do think you really need to own the land to be successful. That doesn't mean you have to farm it, it just means you have the control over who you rent to and what practices you're comfortable with. I also think your chances of success are greater when share-cropping vs cash rent. Expectations also need to be lowered. Don't count on raking in passive income, just hope your investment breaks even and you end up with a valuable, paid-off chunk of land in 30 years.