When social media becomes a dominant channel for the distribution of news, manipulation of the news agenda and news content can be achieved by anyone who is hosting a website with access to social media APIs. Falsehoods masked as legitimate news with the intent to manipulate the public are called Fake News. This type of propaganda is disseminated by sharing of individual social media users. Fake news pose a threat to democracies as they influence the public agenda and contribute to polarization of opinions. To limit the dissemination of fake news, social media websites utilize fact-checking badges to flag possibly fabricated content. It has however not been investigated whether these badges are effective and who responds to them. In a survey study with 120 participants we found little evidence for the effectiveness of such badges. However, believability of news in a social media sites were generally rated rather low.