critical radius of insulation for cylinder is that radius at which the total heat transfer is maximum. It is the contribution of convective resistance and conductive resistance. There arises a situation of insulation thickness when the total combination of resistance is minimum and hence maximum transfer of heat takes place. After this certain radius if we decrease or increase it further then again resistance combination changes and so does the heat transfer. For cylinder critical radius is
2k/h where k is the conductivity of insulation material and h is convective heat transfer coefficient.
For a plane wall no such thickness exist, because on increasing the thickness of insulation only conductive resistance will increase and no effect on convective resistance as the surface area remains same. This is for 1 dimensional wall and for a 3d wall more complicated phenomena exist.