“When you study natural science and the miracles of creation, if you don't turn into a mystic you are not a natural scientist.”

“Der erste Trunk aus dem Becher der Naturwissenschaft macht atheistisch, aber auf dem Grund des Bechers wartet Gott.” in 15 Jahrhunderte Würzburg: e. Stadt u. ihre Geschichte [15 centuries Würzburg. A city and its history] (1979), p. 205, by Heinz Otremba. Otremba does not declare his source, and the quote per se cannot be found in Heisenberg's published works.
The journalist Eike Christian Hirsch PhD, a personal acquaintance of Heisenberg, whom he interviewed for his 1981 book Expedition in die Glaubenswelt, claimed in de.wikiquote.org on 22 June 2015, that the content and style of the quote was completely foreign to Heisenberg's convictions and the way he used to express himself, and that Heisenberg's children, Dr. Maria Hirsch and Prof. Dr. Martin Heisenberg, did not recognize their father in this quote.
Statements similar to the quote were made by Francis Bacon, in "Of Atheism" (1601): "A little philosophy inclineth man’s mind to atheism; but depth in philosophy bringeth men’s minds about to religion", and Alexander Pope, in "An Essay on Criticism" (1709): "A little learning is a dangerous thing; drink deep, or taste not the Pierian spring: there shallow draughts intoxicate the brain, and drinking largely sobers us again."
There is a passage in a lengthy essay written by Heisenberg in 1942, "Ordnung der Wirklichkeit” ("Reality and Its Order"), published in Collected Works. Section C: Philosophical and Popular Writings. Volume I. Physics and Cognition. 1927-1955 (1984), that parallels the ideas expressed in the quote (albeit in a much expanded form):
"The first thing we could say was simply: 'I believe in God, the Father, the almighty creator of heaven and earth.' The next step — at least for our contemporary consciousness — was doubt. There is no god; there is only an impersonal law that directs the fate of the world according to cause and effect... And yet [today], we may with full confidence place ourselves into the hands of the higher power who, during our lifetime and in the course of the centuries, determines our faith and therewith our world and our fate." (English translation by M.B.Rumscheidt and N. Lukens, available at http://www.heisenbergfamily.org/t-OdW-english.htm)
Carl Friedrich von Weizsäcker, a protégé of Heisenberg, did publish a version of the quote itself in Die Geschichte der Natur (The History of Nature) (1948), appearing to consider it an adage:
"Aus dem Denken gibt es keinen ehrlichen Rückweg in einen naiven Glauben. Nach einem alten Satz trennt uns der erste Schluck aus dem Becher der Erkenntnis von Gott, aber auf dem Grunde des Bechers wartet Gott auf den, der ihn sucht. Wenn es so ist, dann gibt es einen Weg des Denkens, der vorwärts zu religiösen Wahrheiten führt, und nur diesen Weg zu suchen ist lohnend. Wenn es nicht so ist, wird unsere Welt auf die Religion ihre Hoffnungen vergeblich setzen." ("From thinking there is no honest way back into a naive belief. According to an old phrase, the first sip from the cup of knowledge separates us from God, but at the bottom of the cup God is waiting for the one who seeks him. If so, then there is a way of thinking that leads to religious truths, and to seek only that way is rewarding. If it is not so, our world will put its hopes to religion in vain.")
Misattributed

As quoted in The Wright Style (1992) by Carla Lind, p. 3

Letter to his father, expressing his desire to study physics instead of mechanical engineering, as his father had wanted. His father agreed, on the condition that Bhabha pass the Mechanical Engineering Tripos with first class. He did that, and later passed the mathematical Tripos with First class too, as quoted in the "Homi Jehangir Bhabha" profile at the Vigyan Prasar Science Portal.
Context: I seriously say to you that business or job as an engineer is not the thing for me. It is totally foreign to my nature and radically opposed to my temperament and opinions. Physics is my line. I know I shall do great things here. For, each man can do best and excel in only that thing of which he is passionately fond, in which he believes, as I do, that he has the ability to do it, that he is in fact born and destined to do it... I am burning with a desire to do physics. I will and must do it sometime. It is my only ambition. I have no desire to be a "successful" man or the head of a big firm. There are intelligent people who like that and let them do it. … It is no use saying to Beethoven "You must be a scientist bybye for it is great thing" when he did not care two hoots for science; or to Socrates "Be an engineer; it is work of intelligent man." It is not in the nature of things. I therefore earnestly implore you to let me do physics.

Science and the Unseen World (1929), IV, p.47

Source: Attributed in posthumous publications, Einstein and the Poet (1983), p. 117

Look into the stillness

2002

Part 1: "The Creative Mind", §9 (p. 20)
Science and Human Values (1956, 1965)

The Storm Over the University (December 6, 1990)