In this talk I'll give an overview of our contributions to what I call interactive machine learning. Often, interaction in computer science is interpreted as the interaction of humans with the computer but I intend a broader meaning of the interaction of machine learning algorithms with the real world, including but not restricted to humans. Interactions with humans span a broad range where they can be intentional and guided by the human or they can be guided by the computer such that the human is oblivious of the fact that he is being guided. Another example of an interaction with the real world is the use of machine learning algorithms in cyclic discovery processes such as drug design. Important properties of interactive machine learning algorithms include efficiency, effectiveness, responsiveness, and robustness. In the talk I will show how these can be achieved in a variety of interactive contexts.