The watchdog has launched a campaign to "make complaints count in public services" after discovering that many consumers, many of them in more vulnerable groups, do not know who to turn to in the event of a problem.
The poll by Which? revealed that 34% of people who experienced a problem with public services did not complain, with 35% not knowing who to complain to and 39% believing that it simply would not be worth the effort.
Around 39% of those that did complain were not satisfied with the result, while 49% felt their complaint was ignored. Almost 86% who were dissatisfied did not take their complaint any further.
Some 43% mistakenly believed that for an unresolved problem with a GP they would have to go to the Department of Health instead of the Parliamentary and Health Service Ombudsman (PHSO).
Just 52% of people questioned were aware that the PHSO exists compared to 94% who had heard of Trading Standards and 86% who knew of the Financial Services Ombudsman.
The Government announced new measures to strengthen patient feedback in the NHS last year however Which? said that more needs to be done across all public services - including care homes and schools - to encourage people to share their experiences, ultimately improving services.
Which? has called on the Government to create a unified public services ombudsman to help make complaints about public services easier and more efficient.
"Public services are vital to everyone and if something goes wrong it's crucial that people feel it's worth speaking up to help stop the same thing happening again," said Richard Lloyd, Which? executive director, said:
"Barriers to giving feedback must be removed if public services are to deliver the high standards that we all expect.
"We want to see a shake up of the way complaints are handled, to give people the confidence that their complaints count and will trigger action.