"On Parties" (23 January 1792), Papers of James Madison Vol. XIV, pp. 197-8 http://press-pubs.uchicago.edu/founders/documents/v1ch15s50.html
Mr. Madison wished to relieve the sufferers, but was afraid of establishing a dangerous precedent, which might hereafter be perverted to the countenance of purposes very different from those of charity. He acknowledged, for his own part, that he could not undertake to lay his finger on that article in the Federal Constitution which granted a right of Congress of expending, on objects of benevolence, the money of their constituents.
Summation of Madison's remarks (10 January 1794) Annals of Congress, House of Representatives, 3rd Congress, 1st Session, p. 170 http://memory.loc.gov/cgi-bin/ampage?collId=llac&fileName=004/llac004.db&recNum=82; the expense in question was for French refugees from the Haitian Revolution; this summation has been paraphrased as if a direct quote: "I cannot undertake to lay my finger on that article of the Constitution which granted a right to Congress of expending, on objects of benevolence, the money of their constituents."
1790s
Context: In every political society, parties are unavoidable. A difference of interests, real or supposed, is the most natural and fruitful source of them. The great object should be to combat the evil: 1. By establishing a political equality among all. 2. By withholding unnecessary opportunities from a few, to increase the inequality of property, by an immoderate, and especially an unmerited, accumulation of riches. 3. By the silent operation of laws, which, without violating the rights of property, reduce extreme wealth towards a state of mediocrity, and raise extreme indigence towards a state of comfort. 4. By abstaining from measures which operate differently on different interests, and particularly such as favor one interest at the expence of another. 5. By making one party a check on the other, so far as the existence of parties cannot be prevented, nor their views accommodated. If this is not the language of reason, it is that of republicanism.