The protest in Paris, organized by left-wing rebels within the ruling French Socialist party, goes against the line pushed by President Francois Hollande's government that a "no" vote would be a dangerous step towards a possible exit from the euro zone.
Police put the number of protesters in the Bastille Square, a focal point of the 1789 French revolution and a traditional rallying place for protesters, at around 3,000.
Banners included "Austerity basta" and "Solidarity with the Greek people, cancellation of debt".
But a Harris Interactive opinion poll showed two thirds of those questioned in France wanted European countries to stop lending to Greece and 53 percent believe Greece will not pay back any its debts. Only 4 percent think it will pay everything back.
Earlier, Finance Minister Michel Sapin said a "yes" vote on Sunday would allow dialogue with Greece on a possible accord to restart. He insisted that while further debt relief could be considered, it was important that Athens paid back what it owed.
