“The formation of scales and of the web of harmony is a product of artistic invention, and is in no way given by the natural structure or by the natural behaviour of our hearing, as used to be generally maintained hitherto.”

"The Edict of Chandigarh," 1959

from On the Method of Theoretical Physics, p. 183. The Herbert Spencer Lecture, delivered at Oxford (10 June 1933). Quoted in Einstein's Philosophy of Science http://plato.stanford.edu/entries/einstein-philscience/
1930s
Context: Our experience hitherto justifies us in trusting that nature is the realization of the simplest that is mathematically conceivable. I am convinced that purely mathematical construction enables us to find those concepts and those lawlike connections between them that provide the key to the understanding of natural phenomena. Useful mathematical concepts may well be suggested by experience, but in no way can they be derived from it. Experience naturally remains the sole criterion of the usefulness of a mathematical construction for physics. But the actual creative principle lies in mathematics. Thus, in a certain sense, I take it to be true that pure thought can grasp the real, as the ancients had dreamed.

Source: The Art of the Dance (1928), p. 78.
Context: The harmony of music exists equally with the harmony of movement in nature.
Man has not invented the harmony of music. It is one of the underlying principles of life. Neither could the harmony of movement be invented: it is essential to draw one’s conception of it from Nature herself, and to see the rhythm of human movement from the rhythm of water in motion, from the blowing of the winds on the world, in all the earth’s movements, in the motions of animals, fish, birds, reptiles, and even in primitive man, whose body still moved in harmony with nature….. All the movements of the earth follow the lines of wave motion. Both sound and light travel in waves. The motion of water, winds, trees and plants progresses in waves. The flight of a bird and the movements of all animals follow lines like undulating waves. If then one seeks a point of physical beginning for the movement of the human body, there is a clue in the undulating motion of the wave.

“Unlikely Passages” 1984

Source: Collected Poems (1993), p. 268-269. The Single Hound.

Credo (1965)

Source: Quotes of Paul Cezanne, after 1900, Cézanne, - a Memoir with Conversations, (1897 - 1906), p. 151, in: 'What he told me – I. The motif'

From his "Autobiographische Skizze" (18 April 1955), original German version here http://philoscience.unibe.ch/documents/kursarchiv/WS99/Skizze.pdf. Translation from Subtle is the Lord: The Science and the Life of Albert Einstein by Abraham Pais (1982), p. 131 http://books.google.com/books?id=U2mO4nUunuwC&lpg=PP1&pg=PA131#v=onepage&q&f=false. Pais notes that when he said "during that year", he was referring to some time between October 1895 and early fall 1896.
Variant: "Innovation is not the product of logical thought, even though the final product is tied to a logical structure."
Original German version: Während dieses Jahres in Aarau kam mir die Frage: Wenn man einer Lichtwelle mit Lichtgeschwindigkeit nachläuft, so würde man ein zeitunabhängiges Wellenfeld vor sich haben. So etwas scheint es aber doch nicht zu geben! Dies war das erste kindliche Gedanken-Experiment, das mit der speziellen Relativitätstheorie zu tun hat. Das Erfinden ist kein Werk des logischen Denkens, wenn auch das Endprodukt an die logische Gestalt gebunden ist. ("Autobiographische Skizze", p. 10)
1950s
Context: During that year in Aarau the question came to me: If one runs after a light wave with [a velocity equal to the] light velocity, then one would encounter a time-independent wavefield. However, something like that does not seem to exist! This was the first juvenile thought experiment which has to do with the special theory of relativity. Invention is not the product of logical thought, even though the final product is tied to a logical structure.

In a letter to Rudolph Steiner, c. 1921-23; as quoted in Abstract Painting, Michel Seuphor, Dell Publishing Co., 1964, p. 85
1920's