Yes I believe that silence is giving people consent to do as they want. When you have many small groups speaking out about this and that plus the large majority keeping quiet guess what. The small groups are going to get what they want. Then you have the masses getting angry? Why? IF you the masses had spoken up before hand their would not even be a issue. Stop your whining now and live with what you let pass.
It is the same for the common criminal. As long as people watch them and not do not do anything your letting them get bolder and more outrageous because you are helping them by not speaking up.