It really depends on the situation and person.
1. Yes, Man Problems
Many times people just say 'yes' to please the other people. In that case, saying 'yes', isn't really that powerful. They are doing everyone a disservice.
2. Anxiety, Afraid to allow possibilities
People often say 'No' when they're afraid of possibilities and change. As with 'Yes, man' people, there's usually a sense of anxiety involved.
I think what is truly empowering is when you listen to your inner voice and are speaking what you want. When you come to a decision on your own and use your words to carve the rest of the life you still have left to live.
We only have 4000 weeks to live (to 90ish).
The first 1k don't count, we're still figuring out life.
The next 1k are our prime years
The 3rd quarter is where many people come to realize
The last 1k are where we're most dependent on others and our previous actions.
Use our time wisely. Each 'YES' and 'NO' carry some weight.
The little decision build up to bigger ones.
Whether we asked the right person for their phone number, bought bitcoin/your-favorite-stock when it was cheap, or chiseled a 6pack into your core; the results were the product of many small decisions that came up to where you are now.
True power comes from choice, so Choose 'Yes' and 'No' when you mean it and it's right for you.