Training is slow? Here's my secret weapon
Using cloud GPU instances and Donkey Car Console to reduce model training time.
The first few times I experimented with Donkey Car at our local meetup, I used my Mac to train the model. Even on a recent machine, training with 3,000 to 5,000 images took 20 to 30 minutes. That is manageable if the first training run succeeds, but troubleshooting can consume an entire meetup.
I started using powerful cloud servers to train faster. On AWS, spot instances can provide a capable machine at a relatively low cost. To simplify creating the instance, uploading tubs and downloading the trained model, I created Donkey Car Console.
Donkey Car Console is a web application that runs directly on the Raspberry Pi. It can:
- Upload tubs, train with a selected instance type and download the model back to the Pi.
- Manage tubs and models through a graphical interface.
- Delete unusable tubs and add notes to useful tubs or models.
Donkey Car Console is open source. You pay only for the cloud computing time you use. See the project on GitHub for setup instructions.
Security note: Donkey Car Console requires an AWS key. Protect the key, configure a budget alarm and disable the key when it is no longer needed.
How about Google Colab?
Other Donkey Car users have suggested Google Colab as another way to train a model in the cloud. It provides GPU resources and may be a useful alternative for experimentation.
Happy hacking!
Jonathan