“The witty woman is a tragic figure in American life. Wit destroys eroticism and eroticism destroys wit, so women must choose between taking lovers and taking no prisoners.”

A College Breakfast-party, reported in Bartlett's Familiar Quotations, 10th ed. (1919)
Scenes of Clerical Life (1858)

Variant: Better a witty fool, than a foolish wit.
Source: Twelfth Night

Source: Henry IV, Part 2

Actually from 1969 leaflet by The Feminists, "Women: Do You Know the Facts About Marriage?". Its text was reprinted in the anthology Sisterhood is Powerful edited by Robin Morgan.
Misattributed

Bk. 2, Pt.. 3, Ch. 4: The Lesbian. P. 445 (1974 Vintage edition)
The Second Sex (1949)

Source: Villette (1853), Chapter XXXVII: Sunshine

Aphorism 26, as translated in Dialogue on Poetry and Literary Aphorisms (1968), p. 151
Variant translation:
Wit is the appearance, the external flash, of fantasy. Hence its divinity and the similarity to the wit of mysticism.
As translated in The Early Political Writings of the German Romantics (1996) edited by Frederick C. Beiser, p. 131

Literary Remains

Variant: It takes years to build up, it takes moments to destroy.
Source: Hardboiled Wonderland and the End of the World (1985), Chapter 19: Hamburgers,Skyline and Deadline
Source: Hard-Boiled Wonderland and the End of the World
Context: Once again, life had a lesson to teach me: It takes years to build up, it takes moments to destroy. Sure, I'd gotten tired of this tiny space, but I'd had a good home here. In the time it takes to swill two cans of beer, all had had sublimed like morning mist. My job, my whiskey, my peace and quiet, my solitude, my Sormerset Maugham, and John Ford collections-all of it trashed and worthless.