It will probably come down to interpreting the agreement, but if it was not drawn up by a lawyer, or was drawn up by a lawyer who wasn't clear about what was intended then there could be lots of different interpretations. I would say: legal fights over inheritances, much like legal fights over property boundaries, have the potential to cost more than any of the properties involved are worth. Don't go to court on this if you have any other options at all, including sucking it up and paying the costs of removal yourself.
There are a few options for how the agreement might be interpreted, and there are two separate legal questions, one as to rights to the land and one as to rights to the bins, and these questions interact with each other.
The agreement about the use of the land could be a lease, and specifically a "building lease", in which your uncle rented the land with your grandfather's permission to build the grain bins. In that case at the end of the agreement the bins would probably belong to the owner of the land, although the lease agreement might include conditions about maintenance or paying for removal. Generally a lease needs to be for a defined length of time, so if the agreement did not have an end date it might not be a lease. If there is a lease which is still in force then it passes to your uncle's estate.
The agreement might be just a "licence to occupy", rather than a lease. If it is, then it was probably personal to your uncle and terminated on his death.
The other question is whether the bins are "fixtures" in land law, the answer to which is complicated but mainly depends on whether the bins are fixed to the land or just sitting on it. If they are fixtures, it could be argued that they belong with the land, whether there was a lease or a licence, unless the agreement specifies otherwise. If they are not fixtures, they could be the personal property of your uncle's estate, and if there is no longer any agreement in force about the occupation of the land then the landowner could require them to be removed.
tl;dr: your lawyer is right: it's complicated.
Also, this is off the cuff advice from an internet stranger and is in no way to be relied upon as legal advice.