Search

Sample compression schemes were first proposed by Littlestone and Warmuth in 1986. Undirected graphical model is a powerful tool for classification in statistical learning. In this paper, we consider labelled compression schemes for concept classes induced by discrete undirected graphical models. For the undirected graph of two vertices with no edge, where one vertex takes two values and the other vertex can take any finite number of values, we propose an algorithm to establish a labelled compression scheme of size VC dimension of associated concept class. Further, we extend the result to other two types of undirected graphical models and show the existence of labelled compression schemes of size VC dimension for induced concept classes. The work of this paper makes a step forward in solving sample compression problem for concept class induced by a general discrete undirected graphical model.

One of the open problems in machine learning is whether any set-family of VCdimension d admits a sample compression scheme of size O(d). In this paper, we study this problem for balls in graphs. For a ball B =Br(x) of a graph G= (V,E), a realizable sample for B is a signed subset X = (X+,X) of V such that B contains X+ and is disjoint from X. A proper sample compression scheme of size k consists of a compressor and a reconstructor. The compressor maps any realizable sample X to a subsample X\prime of size at most k. The reconstructor maps each such subsample X\prime to a ball B\prime of G such that B\prime includes X+ and is disjoint from X. For balls of arbitrary radius r, we design proper labeled sample compression schemes of size 2 for trees, of size 3 for cycles, of size 4 for interval graphs, of size 6 for trees of cycles, and of size 22 for cube-free median graphs. For balls of a given radius, we design proper labeled sample compression schemes of size 2 for trees and of size 4 for interval graphs. We also design approximate sample compression schemes of size 2 for balls of\delta-hyperbolic graphs. [ABSTRACT FROM AUTHOR]

Combining two results from machine learning theory we prove that a formula is NIP if and only if it satisfies uniform definability of types over finite sets (UDTFS). This settles a conjecture of Laskowski. [ABSTRACT FROM AUTHOR]

Sample compression schemes were defined by Littlestone and Warmuth (1986) as an abstraction of the structure underlying many learning algorithms. Roughly speaking, a sample compression scheme of size k means that given an arbitrary list of labeled examples, one can retain only k of them in a way that allows us to recover the labels of all other examples in the list. They showed that compression implies probably approximately correct learnability for binary-labeled classes and asked whether the other direction holds. We answer their question and show that every concept class C with VC dimension d has a sample compression scheme of size exponential in d. [ABSTRACT FROM AUTHOR]

Sample compression schemes are schemes for “encoding” a set of examples in a small subset of examples. The long-standing open sample compression conjecture states that, for any concept class C of VC-dimension d , there is a sample compression scheme in which samples for concepts in C are compressed to samples of size at most d . We show that every order over C induces a special type of sample compression scheme for C , which we call order compression scheme. It turns out that order compression schemes can compress to samples of size at most d if C is maximum, intersection-closed, a Dudley class, or of VC-dimension 1 – and thus in most cases for which the sample compression conjecture is known to be true. Since order compression schemes are much simpler than sample compression schemes in general, their study seems to be a promising step towards resolving the sample compression conjecture. We reveal a number of fundamental properties of order compression schemes, which are helpful in such a study. In particular, order compression schemes exhibit interesting graph-theoretic properties as well as connections to the theory of learning from teachers. To obtain small compressed sets, order compression schemes for a concept class C must often use a proper superset H ⊃ C as a hypothesis space. We thus further compare order compression schemes for C to order compression schemes for such hypothesis spaces, leading to a study of a number of mutually related combinatorial parameters specifying compressibility. [ABSTRACT FROM AUTHOR]