Designing a Testing Framework for Transfer Learning Algorithms (Application Paper)

Most works covering the topic of transfer learning propose an algorithm to solve a given domain adaptation problem, then test the algorithm using real-world datasets. A test with a real-world dataset represents a single transfer learning test condition, which partially measures an algorithm's performance. Previous research has placed little emphasis on developing a comprehensive and uniform test for transfer learning algorithms. With this in mind, a test framework is proposed, comprising of distortion profiles which define a comprehensive test suite. The unique contribution of this paper is the definition of a test framework that measures a more complete profile of a transfer learning algorithm's capability, facilitating the identification of relative poor and good performance areas. As a proof of concept, the test framework is used to test a homogeneous transfer learning algorithm. The test framework will be the basis for a number of future applications.