Interpreting mathematical models in general as functions that map parameters to model behaviors (for example, how reaction rates depend on reactant concentrations, or how protein concentrations depend on time), and interpreting such functions geometrically, lead to the field of information geometry: model behaviors now can be represented as a manifold with parameters as its coordinates. Such a perspective leads to conceptual elegance and practical improvements for standard modeling tasks such as parameter estimation and model selection. Importantly, the task of model reduction now becomes approximating a manifold by its boundaries, and the limiting model behaviors at the boundaries also give rise to scientific insight on how collective behaviors of a system emerge from its microscopic mechanisms, a central focus for systems science including systems biology. Realizing the importance of manifold boundaries, the incipient field of information topology attempts a topological characterization of all boundaries of a manifold, and constitutes the starting point of this thesis. We propose a system-component formulation of system models, a class of models that encompass typical systems biology models. The simple formulation greatly organizes our understanding of the boundary structure of a system model: first, it partitions the set of boundaries into component boundaries and emergent boundaries, and explains their origins; second, it explains the origin of structural nonidentifiability and the difficulty of resolving it; third, it helps explain additional features of the boundary structure, namely combinatoriality and symmetry. Next, we interpret structural nonidentifiability geometrically and develop a general method for resolving it using the topological characterization of a manifold; we call the method manifold boundary identification method. Lastly, these results naturally lead to an algorithm for constructing the topological characterization of a system model. Applying these general insights to kinetic models of biochemical reaction networks, especially metabolic networks, yield some partial results. First, we provide complete topological characterizations of some rate laws commonly used in systems biology such as Michaelis-Menten, which pave way for similar characterizations of system models constructed using the rate laws. Second, we formalize the common system behaviors of metabolic networks, cast them into the system-component formulation, and characterize the resulting functional compositional structure, which for some behaviors translates to improved algorithms of boundary exploration for systems models. Third, focusing on the metabolic behaviors of how network fluxes depend on external metabolite concentrations, which are commonly measured in metabolic research, we characterize some of its mathematical structures, namely similarity, which gives metabolic networks interpretations as generalized reactions with the system behaviors as generalized rate laws, and modularity, which decomposes a metabolic network into interpretable modules in a formal and precise way; these insights, together with the ideas and techniques from information geometry and information topology, hold the promise of shedding light on some of long-standing problems in metabolic modeling.