“If I were not an atheist, I would believe in a God who would choose to save people on the basis of the totality of their lives and not the pattern of their words.”

Where All Roads Lead (1922); this is often misquoted as "If there were no God, there would be no atheists."
Context: Atheism is, I suppose, the supreme example of a simple faith. The man says there is no God; if he really says it in his heart, he is a certain sort of man so designated in Scripture [i. e. a fool, Ps 53:2]. But, anyhow, when he has said it, he has said it; and there seems to be no more to be said. The conversation seems likely to languish. The truth is that the atmosphere of excitement, by which the atheist lived, was an atmosphere of thrilled and shuddering theism, and not of atheism at all; it was an atmosphere of defiance and not of denial. Irreverence is a very servile parasite of reverence; and has starved with its starving lord. After this first fuss about the merely aesthetic effect of blasphemy, the whole thing vanishes into its own void. If there were not God, there would be no atheists.

Source: Life Itself : A Memoir (2011), Ch. 54 : How I Believe In God
Context: Over the high school years, my belief in the likelihood of a God disappeared. I kept this to myself. I never discussed it with my parents. My father in any event was a nonpracticing Lutheran, until a deathbed conversion that rather disappointed me. I’m sure he agreed to it for my mother’s sake. Did I start calling myself an agnostic or an atheist? No, and I still don’t. I avoid that because I don’t want to provide a category that people can apply to me. Those who say that “believer” and “atheist” are concrete categories do violence to the mystery we must be humble enough to confess. I would not want my convictions reduced to a word.

<!-- p. 76 -->
In Search of the Miraculous (1949)

As quoted in Against the Faith (1985) by Jim Herrick, p. 75
Variant translation: It is very important not to mistake hemlock for parsley, but to believe or not believe in God is not important at all.

How to Have a Life-Style

Taste of My Life BBC2 (2008)

TV appearances
Context: Eddie Izzard: What if someone comes to you in the middle of the Second World War and says, "Do you have any Jewish people in your house?" and you do have them. That would be a lie. That would be disrespectful to Hitler.
Christine O'Donnell: I believe if I were in that situation, God would provide a way to do the right thing righteously. I believe that!
Bill Maher: God is not there. Hitler's there and you're there.
Christine O'Donnell: You never have to practice deception. God always provides a way out.

A Christmas Sermon (1890)