“I want to emphasize the necessity for a sound mathematical basis for any fundamental physical theory. Any philosophical ideas that one may have play only a subordinate role. Unless such ideas have a mathematical basis they will be ineffective.”

On Human Nature (1978), Ch.3 Development

Ch. 1, Mathematical Elegance as a Driving Force, p. 62 https://books.google.com/books?id=T09kCwAAQBAJ&pg=PA62.
Fashion, Faith, and Fantasy in the New Physics of the Universe (2016)

Quoted by C. S. Hastings in "Biographical Memoir of Josiah Willard Gibbs 1839-1903," National Academy of Sciences Biographical Memoirs, vol. VI (Washington, D.C.: National Academy of Sciences, 1909), p. 390. Complete memoir http://books.nap.edu/html/biomems/jgibbs.pdf
Attributed

The Evolution of the Physicist's Picture of Nature (1963)
Context: It seems to be one of the fundamental features of nature that fundamental physical laws are described in terms of a mathematical theory of great beauty and power, needing quite a high standard of mathematics for one to understand it. You may wonder: Why is nature constructed along these lines? One can only answer that our present knowledge seems to show that nature is so constructed. We simply have to accept it. One could perhaps describe the situation by saying that God is a mathematician of a very high order, and He used very advanced mathematics in constructing the universe. Our feeble attempts at mathematics enable us to understand a bit of the universe, and as we proceed to develop higher and higher mathematics we can hope to understand the universe better.

The Nature of the Physical World (1928)
Context: The idealistic tinge in my conception of the physical world arose out of mathematical researches on the relativity theory. In so far as I had any earlier philosophical views, they were of an entirely different complexion.
From the beginning I have been doubtful whether it was desirable for a scientist to venture so far into extra-scientific territory. The primary justification for such an expedition is that it may afford a better view of his own scientific domain.

Preface http://www-groups.dcs.st-and.ac.uk/~history/Extras/Eddington_Gifford.html

Book III, Ch. 9, sec. 4
An Essay Concerning Human Understanding (1689)

[Jon Fripp, Michael Fripp, Deborah Fripp, Speaking of Science: Notable Quotes on Science, Engineering, and the Environment, https://books.google.com/books?id=44ihCUS1XQMC&pg=PA45, 2000, Newnes, 978-1-878707-51-2, 45]

Source: The Revolution of Nihilism: Warning to the West (1939), p. 253

Introduction
Being and Event (1988)