“Only the scum of the population do it”
As quoted in Roughing It http://www.friesian.com/quotes.htm (1872).
Roughing It (1872)
Context: No California gentleman or lady ever abuses or oppresses a Chinaman, under any circumstances, an explanation that seems to be much needed in the east. Only the scum of the population do it; they and their children. They, and, naturally and consistently, the policemen and politicians, likewise, for these are the dust-licking pimps and slaves of the scum, there as well as elsewhere in America.