Arne Svenson, a New York photographer, was sued by his neighbours after
photographing them in their homes without their permission and exhibiting
them in a nearby gallery.
However, a court ruled that under the First Amendment of the U.S. constitution, it would be a breach of his right to freedom of speech and that this art doesn't need consent.
It should be noted that none of the neighbours were identifiable.
Would you be happy to be photographed surreptitiously in your home and have the photographs published in the name of art?
The Guardian has an article discussing this issue here.