In case of inferior goods which in this case is potatoes, the substituion effect will work in opposite direction to the income effect. When price of potatoes rises, its negative substituion effect will tend to reduce the quantity purchased, while the income effect will tend to increase the quantity purchased. But normally it happens that negative substituion effect of change in price is not large enough to outweigh the income effect.
This is so because a consumer spends a very small proportion of his income on a single commodity and when price of a commodity rises, a very little income is released. In other words substituion effect even when negative is generally too weak to outweigh the income effect.