“Much of the skill of the true mathematical physicist and of the mathematical astronomer consists in the power of adapting methods and results carried out on an exact mathematical basis to obtain approximations sufficient for the purposes of physical measurements. It might perhaps be thought that a scheme of Mathematics on a frankly approximative basis would be sufficient for all the practical purposes of application in Physics, Engineering Science, and Astronomy, and no doubt it would be possible to develop, to some extent at least, a species of Mathematics on these lines. Such a system would, however, involve an intolerable awkwardness and prolixity in the statements of results, especially in view of the fact that the degree of approximation necessary for various purposes is very different, and thus that unassigned grades of approximation would have to be provided for. Moreover, the mathematician working on these lines would be cut off from the chief sources of inspiration, the ideals of exactitude and logical rigour, as well as from one of his most indispensable guides to discovery, symmetry, and permanence of mathematical form. The history of the actual movements of mathematical thought through the centuries shows that these ideals are the very life-blood of the science, and warrants the conclusion that a constant striving toward their attainment is an absolutely essential condition of vigorous growth. These ideals have their roots in irresistible impulses and deep-seated needs of the human mind, manifested in its efforts to introduce intelligibility in certain great domains of the world of thought.”

The Mathematical Foundations of Quantum Theory (1978)

Source: The Fabric of the Cosmos: Space, Time, and the Texture of Reality

§ 2.
Linear Associative Algebra (1882)
Context: The branches of mathematics are as various as the sciences to which they belong, and each subject of physical enquiry has its appropriate mathematics. In every form of material manifestation, there is a corresponding form of human thought, so that the human mind is as wide in its range of thought as the physical universe in which it thinks.

[Jon Fripp, Michael Fripp, Deborah Fripp, Speaking of Science: Notable Quotes on Science, Engineering, and the Environment, https://books.google.com/books?id=44ihCUS1XQMC&pg=PA45, 2000, Newnes, 978-1-878707-51-2, 45]

Lecture on "Electrical Units of Measurement" (3 May 1883), published in Popular Lectures Vol. I, p. 73, as quoted in The Life of Lord Kelvin (1910) by Silvanus Phillips Thompson

"The Mathematician", in The Works of the Mind (1947) edited by R. B. Heywood, University of Chicago Press, Chicago
Context: I think that it is a relatively good approximation to truth — which is much too complicated to allow anything but approximations — that mathematical ideas originate in empirics. But, once they are conceived, the subject begins to live a peculiar life of its own and is … governed by almost entirely aesthetical motivations. In other words, at a great distance from its empirical source, or after much "abstract" inbreeding, a mathematical subject is in danger of degeneration. Whenever this stage is reached the only remedy seems to me to be the rejuvenating return to the source: the reinjection of more or less directly empirical ideas.

Richard Courant in: The Parsimonious Universe, Stefan Hildebrandt & Anthony Tromba, Springer-Verlag, 1996, page 148

Source: Systems Engineering Tools, (1965), p. 111 as cited in

Methods of Mathematics Applied to Calculus, Probability, and Statistics (1985)