When working in dog rescue, I've heard these words from potential adopters a few times. While it's fantastic that the speakers had the dog's best interest at heart, at times this fear of harming the dog's well-being actually prevented it from being placed in a great home.
What I've noticed from fostering, dog sitting, mentoring adopters and working with my own dogs is that we fall hopelessly in love with them much faster than they do with us. Don't get me wrong, they do love us. But it takes a while for them to form an attachment. When returned in the early stages of adoption, the dog won't really miss you. Should you come back for him, he'll be very happy to see you. But he wasn't pining for you or depressed.
Dogs that are social butterflies are especially happy anywhere they are as long as there are people there. I've seen young dogs returned to our rescue group and placed into foster homes and they don't seem at all sad or heartbroken.
The only times I've seen heartbroken dogs have been those that were with a family for several years before being returned. I'd give a rough estimate that if you've had the dog less than a year, you can return him without upsetting him. But if you've had the dog longer than a year, it's better if you keep him. (Watch for another article where I explain exceptions to this guideline.)
This is why fostering works so well for dogs. If you've been considering adopting or fostering but held back because of this fear, don't worry about it! Work with the rescue group to select the best dog for your family and if things don't go as well as hoped, it's not at all cruel to return the dog. Another home will be found for him and he'll likely just remember you as "those nice people I stayed with for a while"!