Religious and ethnic minorities often face partisan treatment with regard to provision of public goods. This may be due to discriminatory practices or historical antecedents, like caste divides in India. We measure access to public goods in eleven districts of West Bengal in India where rural concentration of religious minorities, namely Muslims is higher than the state and country-level averages. We look at the contemporary socio-economic conditions of religious minorities in the state and the outcome regarding access to public good. The evidence as presented in this paper is based on cross-section data collected between Censuses of India. This is unavailable in the extant literature. Our unit of analysis for the econometric exercise is the village. We calculated average values of the relevant variables for each village from household responses, and the aggregate village level information from the village survey data. To this end, we adopted a stratified multistage sampling design in which households are the targeted sampling units chosen from rural areas only. The first stage units are the 2001 Census villages constituting the primary sampling units. The development blocks as comprising of Census villages have been classified into three strata on the basis of the percentage of Muslim population. Subsequently, using Generalized Linear Models we find evidence of strong horizontal inequality against Muslims in terms of access to public goods. We estimated the access to targeted and non-targeted public goods such as drinking water, distance to hospitals, distance and quality of educational facilities, transport, condition of roads, etc, based on the level of concentration of Muslim population. We use percentage of Muslim population at the village level and the highest concentration dummy as key explanatory variables. Provision of public goods seems to have a negative relation with the rise in concentration of minority population. Further, to decompose the outcome variables in terms of explained (viz. education and occupation) and unexplained (viz. discrimination) variations, we use Blinder-Oaxaca technique. In particular, the minority population in concentrated districts face poorer access to infrastructure, health and transport facilities. As policy, the district level plans need to address deficits in identifiable pockets of underdevelopment with respect to some of the variables discussed here. Moreover, the persistence of information gaps and poor implementation of policies in several parts, including the region under study, despite elaborate decentralization of governance in the country needs to be addressed from institutional perspectives in future analyses.