This situation could turn into a blessing in disguise.
First, you need to adopt a new attitude. You can USE this crochety old lady to your advantage in training your puppy.
That's step one.
First, while in the house, teach your puppy the "With Me" game. Have a pocketful of yummy treats (tiny bits of chicken, hot dog, cheese) and a small squeeky toy (small enough to put in your pocket or tuck in your shirt). Put a longish lead on him and then just stand still. Wait for him to start to wander away from you and then explode with glee, patting your leg and/or squeeking the toy while dancing side to side ......and say "With Me!" in a very upbeat, animated voice. You will become very interesting! As soon as he turns and starts to come to you, hop backwards a few steps, continuing the antics until he gets to you. IMMEDIATELY treat him and tell him "GOOD With Me!" "Good puppy!" and give him a pat.......and then release him with an understated "ohhh-kay" (said like "oh well, game over") and then start the game again by standing still and doing nothing. It will take longer for him to wander away once he catches on to the reward, so just wait him out until he finds something else to do and wanders away again......then repeat the glee, dance, "With Me!", and reward.
Once he catches on, take it outside and do it there. Then, look forward to crotchety lady's appearance on the scene, and immediately go into the "With Me!" game. USE her to help your puppy learn.
This game is good off lead, too, but I would keep him on lead to start and for quite some time. Make sure you are prepared to play the game each time you go outside, since you don't know when she's going to be there or not. Have the lead on, pocketful of tasty treats, and a squeeky toy. Be prepared to USE her presence as an opportunity to practice the "With Me!" game, rather than feel like you need to hide when she shows up. It's a completely different mindset to welcome her presence (not her but her presence) as a teaching opportunity than it is to feel self-conscious that your puppy might not be behaving up to her standards when her eyes are on you. That's what you need to do. USE her. Both you and your puppy will learn that she is of no consequence in the scheme of things, other than a training aid.
And.....yes, classes ALWAYS help.
I had to deal with this with a neighbor's lives outside-fence fighting-24 hours a day barking dog that they will do nothing about (other than throw lit cigarettes over the wall then I asked them to please put their dog up when he started up). My dog thought it was great to respond in kind to this dog, so I understand your frustration. I felt like I had no peace, even in my own backyard because every time Chili and I went out there I had to spend the entire time working with her instead of anything else.
AND....it took awhile (she's 2-1/2), but it was SO worth the effort. My dog can now do ALL her obedience exercises perfectly, including the upper level exercises, without giving that dog more than a slight head turn and right back to business acknowledgement. Even better, she can wander my yard without being reactive to him - she's learned he means nothing. For a dog (mine) that would be very happy to oblige that dog and spend her day challenging back if she had been allowed to, that is saying something.....and I'm VERY proud of her for her accomplishment.
So....I say turn lemons into lemonade!