“Lest, once more wandering from that heaven,
I fall on some base heart unblest,
Faithless to thee, false, unforgiven,
And lose my everlasting rest.”

Reported in Josiah Hotchkiss Gilbert, Dictionary of Burning Words of Brilliant Writers (1895), p. 247.

(19th May 1827) Genius
The London Literary Gazette, 1827

Nearer, my God, to Thee (c. 1841)

From Sonnet III: "Oh, Think not I am faithful to a vow!", A Few Figs from Thistles (1922) <!-- Not sure whether this appears in the 1920 edition. -->
Context: But you are mobile as the veering air,
And all your charms more changeful than the tide,
Wherefore to be inconstant is no care:
I have but to continue at your side.
So wanton, light and false, my love, are you,
I am most faithless when I most am true.

tr. Phillip H. De Lacy and Benedict Einarson. Cf. full quotation at Leonard p. 54-55 https://books.google.com/books?id=omUTAAAAYAAJ&pg=PA54#v=onepage&q&f=false
fr. 115, as paraphrased in Plutarch's Moralia
Purifications
Context: A law there is, an oracle of Doom, Of old enacted by the assembled gods, That if a Daemon—such as live for ages— Defile himself with foul and sinful murder, He must for seasons thrice ten thousand roam Far from the Blest; such is the path I tread, I too a wanderer and exile from heaven.

Source: Moby-Dick or, The Whale

"The Dirge of the Sea" (April 1891)
Context: Years! Years, ye shall mix with me!
Ye shall grow a part
Of the laughing Sea;
Of the moaning heart
Of the glittered wave
Of the sun-gleam's dart
In the ocean-grave. Fair, cold, and faithless wert thou, my own!
For that I love
Thy heart of stone!
From the heights above
To the depths below,
Where dread things move, There is naught can show
A life so trustless! Proud be thy crown!
Ruthless, like none, save the Sea, alone!

Poemː God
Reported in Josiah Hotchkiss Gilbert, Dictionary of Burning Words of Brilliant Writers (1895), p. 283.

(10th April 1824) Love in Absence
The London Literary Gazette, 1824