“If I have had any success in mathematical physics, it is, I think, because I have been able to dodge mathematical difficulties.”

"Newton's Principia" in 300 Years of Gravitation. (1987) by S. W. Hawking and W. Israel, p. 4

Source: The Development of Mathematics (1940), p. 283
Context: The mistakes and unresolved difficulties of the past in mathematics have always been the opportunities of its future; and should analysis ever appear to be without or blemish, its perfection might only be that of death.

[Oliver Heaviside (1850-1927) - Physical mathematician, http://teamat.oxfordjournals.org/content/2/2/55.extract, https://www.gwern.net/docs/science/1983-edge.pdf, Teaching mathematics and its applications, Oxford Journals, 2, 2, 55-61, 1983, DA Edge]
This quote cannot be found in Heaviside's corpus, Edge provides no reference, the quote first appears around the 1940s attributed to Heaviside without any references. The quote is actually a composite of a modified sentence from Electromagnetic Theory I https://archive.org/details/electromagnetict02heavrich/page/8/mode/2up (changing 'dinner' to 'eat'), a section header & later sentence from Electromagnetic Theory II https://archive.org/details/electromagnetict02heavrich/page/4/mode/2up, and the paraphrase of Heaviside's views by Carslaw 1928 https://www.gwern.net/docs/math/1928-carslaw.pdf ("Operational Methods in Mathematical Physics"), respectively:
"Nor is the matter an unpractical one. I suppose all workers in mathematical physics have noticed how the mathematics seems made for the physics, the latter suggesting the former, and that practical ways of working arise naturally. This is really the case with resistance operators. It is a fact that their use frequently effects great simplifications, and the avoidance of complicated evaluations of definite integrals. But then the rigorous logic of the matter is not plain! Well, what of that? Shall I refuse my dinner because I do not fully understand the process of digestion? No, not if I am satisfied with the result. Now a physicist may in like manner employ unrigorous processes with satisfaction and usefulness if he, by the application of tests, satisfies himself of the accuracy of his results. At the same time he may be fully aware of his want of infallibility, and that his investigations are largely of an experimental character, and may be repellent to unsympathetically constituted mathematicians accustomed to a different kind of work."
"Rigorous Mathematics is Narrow, Physical Mathematics Bold And Broad. § 224. Now, mathematics being fundamentally an experimental science, like any other, it is clear that the Science of Nature might be studied as a whole, the properties of space along with the properties of the matter found moving about therein. This would be very comprehensive, but I do not suppose that it would be generally practicable, though possibly the best course for a large-minded man. Nevertheless, it is greatly to the advantage of a student of physics that he should pick up his mathematics along with his physics, if he can. For then the one will fit the other. This is the natural way, pursued by the creators of analysis. If the student does not pick up so much logical mathematics of a formal kind (commonsense logic is inherited and experiential, as the mind and its ways have grown to harmonise with external Nature), he will, at any rate, get on in a manner suitable for progress in his physical studies. To have to stop to formulate rigorous demonstrations would put a stop to most physico-mathematical inquiries. There is no end to the subtleties involved in rigorous demonstrations, especially, of course, when you go off the beaten track. And the most rigorous demonstration may be found later to contain some flaw, so that exceptions and reservations have to be added. Now, in working out physical problems there should be, in the first place, no pretence of rigorous formalism. The physics will guide the physicist along somehow to useful and important results, by the constant union of physical and geometrical or analytical ideas. The practice of eliminating the physics by reducing a problem to a purely mathematical exercise should be avoided as much as possible. The physics should be carried on right through, to give life and reality to the problem, and to obtain the great assistance which the physics gives to the mathematics. This cannot always be done, especially in details involving much calculation, but the general principle should be carried out as much as possible, with particular attention to dynamical ideas. No mathematical purist could ever do the work involved in Maxwell's treatise. He might have all the mathematics, and much more, but it would be to no purpose, as he could not put it together without the physical guidance. This is in no way to his discredit, but only illustrates different ways of thought."
"§ 2. Heaviside himself hardly claimed that he had 'proved' his operational method of solving these partial differential equations to be valid. With him [Cf. loc. cit., p. 4. [Electromagnetic Theory, by Oliver Heaviside, vol. 2, p. 13, 1899.]] mathematics was of two kinds: Rigorous and Physical. The former was Narrow: the latter Bold and Broad. And the thing that mattered was that the Bold and Broad Mathematics got the results. "To have to stop to formulate rigorous demonstrations would put a stop to most physico-mathematical enquiries." Only the purist had to be sure of the validity of the processes employed."
Apocryphal

Interview with Dr. P. A. M. Dirac by Thomas S. Kuhn at Dirac's home, Cambridge, England, May 7, 1963 http://www.aip.org/history/ohilist/4575_3.html

Source: The Autobiography of Malcolm X

Source: In a letter addressed to George Stokes dated December 20, 1857, as quoted in Fluid Mechanics in the Next Century https://doi.org/10.1115/1.3101925 (1996), by Mohamed Gad-el-Hak and Mihir Sen.

whatever that may be
Dijkstra (1993) "From my Life" http://www.cs.utexas.edu/users/EWD/transcriptions/EWD11xx/EWD1166.html (EWD 1166).
1990s

Elements de la géométrie de l'infini (1727) as quoted by Amir R. Alexander, Geometrical Landscapes: The Voyages of Discovery and the Transformation of Mathematical Practice (2002) citing Michael S. Mahoney, "Infinitesimals and Transcendent Relations: The Mathematics of Motion in the Late Seventeenth Century" in Reappraisals of the Scientific Revolution, ed. David C. Lindberg, Robert S. Westman (1990)

Original: (fr) Je ne sais si je n’ai déjà dit quelque part que la Mathématique est l’art de donner le même nom à des choses différentes.
Source: Science and Method (1908), Part I. Ch. 2 : The Future of Mathematics, p. 31