Further to add to the already usefully comments, especially about ransom strips, I would buy the title to the Vicarage so you know rights of way or restrictions on that portion of land. It's only £6 for a copy under find a property tab here:
https://eservices.landregistry.gov.uk
Certainly don't speak with the tenants or land owner until you've done as much due diligence because you don't want them to know of your intentions and inflate the price of any possible future lease / freehold.