Ok, I am leaning toward buying it now. I took an appointment with the bank. Although my decision is not final but based on the latest posts, ...he want sheeps, not camels!
But, yes, I can easily afford it. And yes, it's a good incentive for us to just not having neighbors and I wouldn't mind keeping it as it is. I surveyed the market past few days, which I haven't done in years and oh my god! Lots got ridiculously expensive around here. This might actually be a good deal after all. Although there is not that many lots for sale right now, those that could compare to this one were not selling below 175 000$ (but didn't have threes to cut and 200 10-wheelers of dirt to put in) Another point is that I can cut the trees on it myself, have the stomps removed and put a "clean fill accepted" on it and watch the magic happen. I saw one other lot on my street getting completely filled up in 1-2 years that way, for free. All and all, there is a lot of options and I really feel at ease when I think about buying it now, quite excited actually because since the sell sign came up on it last year, I hade quite a discomfort, especially with the work we are putting on our house. That vacant lot is really part of the reason we are so happy here.
For the rental of the finished house, there is a high demand around here for AirB&B style rentals because it's highly touristic and there is a military base close by for longer term rental possibilities to base personnel on rotation, but I looked up for houses for rent and it plays between 1200$ and 1750$ per month witch doesn't seem much to me but I really don't know much about renting houses.