The Rectory, of course, means the Lefroys, and so Tom Lefroy; the same family into which Anna subsequently married.
As to dates, it looks as if it might be around 1750 to me; or would the fanlight make it a bit later? I wonder if Ashe Rectory subsequently became the Ashe House listed by Pevsner, "Hampshire" - the descriptions certainly tally: "Georgian, of brick. Five bays, two storeys, one-bay pediment. Doorway with segmental fanlight." The present church at Ashe dates from 1877-9, and is a Scott, with stained glass by Kempe. But there is a monument to the Rev. Isaac P. G. Lefroy, 1806, "Tablet with an urn. By Gibbs of Basingstoke."
I noted with interest that on the 1790s map, Ashe and Deane were spelt without the final "e". I wonder if (more generally too, of course) the last - now useless - e was added as an assertion of age; or perhaps just a change of fashion - just as Molly, Minny, etc. were not long ago more fashionably spelt Mollie, Minnie.