Letter to Frénicle (1657) Oeuvres de Fermat Vol.II as quoted by Edward Everett Whitford, The Pell Equation http://books.google.com/books?id=L6QKAAAAYAAJ (1912)
Context: There is scarcely any one who states purely arithmetical questions, scarcely any who understands them. Is this not because arithmetic has been treated up to this time geometrically rather than arithmetically? This certainly is indicated by many works ancient and modern. Diophantus himself also indicates this. But he has freed himself from geometry a little more than others have, in that he limits his analysis to rational numbers only; nevertheless the Zetcica of Vieta, in which the methods of Diophantus are extended to continuous magnitude and therefore to geometry, witness the insufficient separation of arithmetic from geometry. Now arithmetic has a special domain of its own, the theory of numbers. This was touched upon but only to a slight degree by Euclid in his Elements, and by those who followed him it has not been sufficiently extended, unless perchance it lies hid in those books of Diophantus which the ravages of time have destroyed. Arithmeticians have now to develop or restore it. To these, that I may lead the way, I propose this theorem to be proved or problem to be solved. If they succeed in discovering the proof or solution, they will acknowledge that questions of this kind are not inferior to the more celebrated ones from geometry either for depth or difficulty or method of proof: Given any number which is not a square, there also exists an infinite number of squares such that when multiplied into the given number and unity is added to the product, the result is a square.
