“To emphasize only the beautiful seems to me to be like a mathematical system that only concerns itself with positive numbers.”

Source: http://www.theguardian.com/science/2014/aug/13/interview-maryam-mirzakhani-fields-medal-winner-mathematician

John D. Barrow, Between Inner and Outer Space: Essays on Science, Art and Philosophy (Oxford University Press, 2000, ISBN 0-192-88041-1, Part 4, ch. 13: Why is the Universe Mathematical? (p. 88). Also found in Barrow's "The Mathematical Universe" http://www.lasalle.edu/~didio/courses/hon462/hon462_assets/mathematical_universe.htm (1989) and The Artful Universe Expanded (Oxford University Press, 2005, ISBN 0-192-80569-X, ch. 5, Player Piano: Hearing by Numbers, p. 250
Misattributed

The Artful Universe (1995)
Context: If a 'religion' is defined to be a system of ideas that contains unprovable statements, then Gödel taught us that mathematics is not only a religion, it is the only religion that can prove itself to be one.<!-- Ch. 5, p. 211

Part 1, Chapter 9 (page 32)
Notes from Underground (1864)
Context: To care only for well-being seems to me positively ill-bred. Whether it's good or bad, it is sometimes very pleasant, too, to smash things.

Prime Obsession: Bernhard Riemann and the Greatest Unsolved Problem in Mathematics (2003)

Earliest source located is the book Brighter than a Thousand Suns: A Personal History of the Atomic Scientists by Robert Jungk (1958), p. 249, which says that Einstein made the comment during "a walk with Ernst Straus, a young mathematician acting as his scientific assistant at Princeton."
Variant: "Equations are more important to me, because politics is for the present, but an equation is something for eternity." From A Briefer History of Time by Stephen Hawking (2005), p. 144 http://books.google.com/books?id=4Y0ZBW19n_YC&lpg=PP1&pg=PA144#v=onepage&q&f=false.
Earlier, Straus recalled the German version of the quote in Helle Zeit, Dunkle Zeit: In Memoriam Albert Einstein (1956) edited by Carl Seelig<!-- Zurich: Europa Verlag -->, p. 71. There the quote was given as Ja, so muß man seine Zeit zwischen der Politik und unseren Gleichungen teilen. Aber unsere Gleichungen sind mir doch viel wichtiger; denn die Politik ist für die Gegenwart da, aber solch eine Gleichung is etwas für die Ewigkeit.
Attributed in posthumous publications
Context: Yes, we now have to divide up our time like that, between politics and our equations. But to me our equations are far more important, for politics are only a matter of present concern. A mathematical equation stands forever.

1900s, "The Study of Mathematics" (November 1907)
Context: Mathematics, rightly viewed, possesses not only truth, but supreme beauty – a beauty cold and austere, like that of sculpture, without appeal to any part of our weaker nature, without the gorgeous trappings of painting or music, yet sublimely pure, and capable of a stern perfection such as only the greatest art can show. The true spirit of delight, the exaltation, the sense of being more than Man, which is the touchstone of highest excellence, is to be found in mathematics as surely as in poetry. What is best in mathematics deserves not merely to be learnt as a task, but to be assimilated as a part of daily thought, and brought again and again before the mind with ever-renewed encouragement.

Source: A Mathematical Theory of Systems Engineering (1967), p. 3.

Cubism was born