[Kandi]

It is so sad to hear what happened to the dog the neighbor was babysitting. I would be devastated if that were to happen to my beloved pet that I entrusted in a friends care.
Absolutely, positively preventable. Where were you? Kids will be kids, and need adult supervision, especially in situations like this. You knew your dog to be dog aggressive. I assume you knew the neighbors were dogsitting, and am sure you know they have a daughter. Common sense tells you the two girls would want to interact if the opportunity arises. Sadly, the small dog got caught in the middle and you were not there to prevent it, as you should have been.
Get a lawyer. Improve your fence--6 foot at least. Wooden would probably be better than chainlink, but no matter what you use. 4.5 feet fences are not enough to contain a large breed dog, be it a Rottweiler or a lab, doesn't matter. Another thing, if your Rottie is still intact, as you say he is, he will jump that fence if he smells a bitch in heat. That fence is not tall enough to even cause him to hesitate. He wants out bad enough, he's gone.
I completely understand your neighbors concerns about the fence and your dog. You are not doing what you need to do for your dogs sake. And, banning the dogs to the fenced in back yard is not good enough either. Dogs are social animals. Assuming you are its alpha, you have basically abandoned that dog to its own devises. Given time, it will be out of your control. Bring the dogs back into the house, and work with them. Continue with training them. They are a part of the family, baby or not. But again, never leave the baby or any other children alone with the dogs. And, never put the baby on the floor with the dog, for him to cuddle. Accidents do happen. You are talking, big dog here. But, such things are preventable if you act responsibly. Yep, your dog might decide the baby's toys are more fun to play with than his, then what?
MO, no reason to put the dog down for just being a dog, but you need to be a responsible dog owner, as do we all. My dogs are never left alone with my nephews and nieces. I have a 13 year old niece and i would still not leave my dogs alone with her. They are never left unsupervised, even in my home. If I am not downstairs, where I can keep an eye on them, they are confined to their crates. It is for their safety and mine.
Again, get a lawyer, and raise the fence. Good luck with the situation.