Learning in Structured Multi-agent Systems with Provable Guarantees