Since Doi Moi (1986), decentralization in Vietnam has been expanded, but still limited to fiscal and administrative rather than political decentralization. From the central perspective, decentralization has undermined the uniformity of national policies and encouraged unhealthy competition among local governments. For local governments, decentralization has not always been accompanied by institutional autonomy and sufficient financial resources. Moreover, there has been a lack of synchronization between central ministries as well as consistency between the different dimensions of decentralization. Finally, the people and businesses have neither been adequately involved nor had sufficient voice in the most important decentralization policies. This article analyses common and cross-cutting issues shared by different dimensions of decentralization in Vietnam since Doi Moi. It shows that serious institutional fragmentation has rendered decentralization ineffective. Given Vietnam’s political economy, the first priority in designing decentralization policy is to overcome this fragmentation and prepare the prerequisites for effective and efficient decentralization.