We study optimal fiscal policy in a small open economy (SOE) with sovereign and private default risk and limited commitment to tax plans. The SOE's government uses linear taxation to fund exogenous expenditures and uses public debt to inter-temporally allocate tax distortions. We characterize a class of environments in which the tax on labor goes to zero in the long run, while the tax on capital income may be non-zero, reversing the standard prediction of the Ramsey tax literature. The zero labor tax is an optimal long run outcome if the economy is subject to sovereign debt constraints and the domestic households are impatient relative to the international interest rate. The front loading of tax distortions allows the economy to build a large (aggregate) debt position in the presence of limited commitment. We show that a similar result holds in a closed economy with imperfect inter-generational altruism, providing a link with the closed-economy literature that has explored disagreement between the government and its citizens regarding inter-temporal tradeoffs.