In this paper we extend the theory of conformal graph directed Markov systems to what we call conformal pseudo-Markov systems. These systems may have a countable infinite set of edges and, unlike graph directed Markov systems, they may also have a countable infinite set of vertices. Our first goal is to develop suitable symbolic dynamics, which we then use to analyze conformal pseudo-Markov systems by giving extensions of various aspects of the thermodynamic formalism and of fractal geometry. Most important, by establishing the existence of a unique conformal measure along with its invariant version, we obtain a generalization of Bowen's formula concerning the Hausdorff dimension of the limit set of a conformal pseudo-Markov system. Here, we also obtain an interesting formula for the closure of the limit set. Finally, we give some applications of our analysis to the theory of Kleinian groups. We show that there exists a rather exotic class of infinitely generated Schottky groups of the second kind (acting on (d + 1)-dimensional hyperbolic space), containing groups with limit sets of Hausdorff dimension equal to any given number t ≤ d, whereas their Poincar´e exponent can be less than any given positive number s t. Moreover, we show that the dissipative part of the limit sets of these groups has further interesting properties.