Complex sequential data in decision-making problems pose a challenge for AI solutions. Most state-of-the-art reinforcement learning algorithms, although actively studied in hope of a solution to this challenge, use the naive action space noise for exploration which limps in practical situations where state-action space becomes huge. We propose a novel auto-curriculum mechanism that divides a problem into smaller sub-problems and let an agent learn from them. This allows us to successfully teach the agent to play football in simulation. Our algorithm takes a step on a path to automatic stock trading, chip design, human-level AI in modern games, and more.