“In spite of all the refinements of civilization that conspired to make art – the dizzying perfection of the string quartet or the sprawling grandeur of Fragonard's canvases – beauty was savage. It was as dangerous and lawless as the earth had been eons before man had one single coherent thought in his head or wrote codes of conduct on tablets of clay. Beauty was a Savage Garden.”

Source: Sentimental Education (1869), Pt. 1, Ch. 4; the most famous portion of this statement is "Exuberance is better than taste..." [Mieux vaut l'exubérance que le goût.]

Progress and Poverty (1879)
Context: I am no sentimental admirer of the savage state. I do not get my ideas of the untutored children of nature from Rousseau, or Chateaubriand, or Cooper. I am conscious of its material and mental poverty, and its low and narrow range. I believe that civilization is not only the natural destiny of man, but the enfranchisement, elevation, and refinement of all his powers, and think that it is only in such moods as may lead him to envy the cud — chewing cattle, that a man who is free to the advantages of civilization could look with regret upon the savage state. But, nevertheless, I think no one who will open his eyes to the facts can resist the conclusion that there are in the heart of our civilization large classes with whom the veriest savage could not afford to exchange. It is my deliberate opinion that if, standing on the threshold of being, one were given the choice of entering life as a Tierra del Fuegan, a black fellow of Australia, an Esquimau in the Arctic Circle, or among the lowest classes in such a highly civilized country as Great Britain, he would make infinitely the better choice in selecting the lot of the savage. For those classes who in the midst of wealth are condemned to want, suffer all the privations of the savage, without his sense of personal freedom; they are condemned to more than his narrowness and littleness, without opportunity for the growth of his rude virtues; if their horizon is wider, it is but to reveal blessings that they cannot enjoy.
There are some to whom this may seem like exaggeration, but it is only because they have never suffered themselves to realize the true condition of those classes upon whom the iron heel of modern civilization presses with full force. As De Tocqueville observes, in one of his letters to Mme. Swetchine, "we so soon become used to the thought of want that we do not feel that an evil which grows greater to the sufferer the longer it lasts becomes less to the observer by the very fact of its duration"; and perhaps the best proof of the justice of this observation is that in cities where there exists a pauper class and a criminal class, where young girls shiver as they sew for bread, and tattered and barefooted children make a home in the streets, money is regularly raised to send missionaries to the heathen! Send missionaries to the heathen! It would be laughable if it were not so sad. Baal no longer stretches forth his hideous, sloping arms; but in Christian lands mothers slay their infants for a burial fee! And I challenge the production from any authentic accounts of savage life of such descriptions of degradation as are to be found in official documents of highly civilized countries — in reports of sanitary commissioners and of inquiries into the condition of the laboring poor.

Source: Walden

Source: Finnikin of the Rock

The lord was James Burnett, Lord Monboddo, (21 August 1773)
See similar debate in Angel.
The Journal of a Tour to the Hebrides with Samuel Johnson, LL.D. (1785)

Source: A Natural History of the Senses (1991)

1911 - 1914, The 'Savages' of Germany' (1912)

A Philosopher: On Wonder And Beauty
Jesus, The Son of Man (1928)
Context: He knew the depth of beauty, He was for ever surprised by its peace and its majesty; and He stood before the earth as the first man had stood before the first day.
We whose senses have been dulled, we gaze in full daylight and yet we do not see. We would cup our ears, but we do not hear; and stretch forth our hands, but we do not touch. And though all the incense of Arabia is burned, we go our way and do not smell.