“Mathematics allows you to see the invisible.”

Source: Language, Truth, and Logic (1936), p. 77.
Context: The principles of logic and mathematics are true simply because we never allow them to be anything else. And the reason for this is that we cannot abandon them without contradicting ourselves, without sinning against the rules which govern the use of language, and so making our utterances self-stultifying. In other words, the truths of logic and mathematics are analytic propositions or tautologies.

"The masked dyer Hakim of Merv" [El tintorero enmascarado Hakim de Merv] Universal History of Infamy (1935); also translated as "Hakim, Masked Dyer of Merv" ( review of "Hakim, Masked Dyer of Merv" http://www.elimae.com/reviews/borges/merv.html)

Source: Tigana

Source: Dragon's Egg (1980), Chapter 7, “Interaction” Section 6 (p. 255)

As quoted by Morris Kline, Mathematical Thought from Ancient to Modern Times (1972)

Source: All the Light We Cannot See

"The Mathematician", in The Works of the Mind (1947) edited by R. B. Heywood, University of Chicago Press, Chicago
Context: I think that it is a relatively good approximation to truth — which is much too complicated to allow anything but approximations — that mathematical ideas originate in empirics. But, once they are conceived, the subject begins to live a peculiar life of its own and is … governed by almost entirely aesthetical motivations. In other words, at a great distance from its empirical source, or after much "abstract" inbreeding, a mathematical subject is in danger of degeneration. Whenever this stage is reached the only remedy seems to me to be the rejuvenating return to the source: the reinjection of more or less directly empirical ideas.

The Untamed Tongue: A Dissenting Dictionary (1990)

As quoted in Stephen Hawking: A Biography (2005) by Kristine Larsen, p. 43