What's a good way to test basic machine learning code?
post by Kenny · 2021-03-11T21:27:15.535Z · LW · GW · 1 commentThis is a question post.
Contents
Answers 3 Kenny 2 Zac Hatfield Dodds None 1 comment
I originally 'scoped' this question based on my inspiration, i.e. a set of example data and their expected answers for various machine learning algorithms or their code components. In other words, a set of test cases that could be used to test implementations of those algorithms or components.
I took the Coursera course Machine Learning (by { Stanford University / Andrew Ng}) and it was great!
I took it because I wanted some 'finger practice' of writing machine learning code and actually running it on my own computer. The course was perfect for that!
One thing in particular that was nice is that the exercises/quizzes/tests that required writing code would have some kind of automated 'endpoint' whereby you could submit your code for testing. (That kind of 'test kata' practice is a really fun way to learn any kind of programming topic, e.g. a new programming language.)
I found these links, on the above course's forum on Coursera, linking to some of the info about the exercises:
The course uses Octave, which was fine.
But I want to basically repeat the course – but using completely different tools. (I want to play with this library in particular.)
What are the best alternatives to the course I took, or similar resources that would be helpful in implementing basic machine learning algorithms, but with arbitrary programming tools?
Answers
Someone suggested these resources:
- UCI Machine Learning Repository
- MLPerf, which is apparently now part of MLCommons
- Stanford DAWN Deep Learning Benchmark (DAWNBench) ·
- deepmind/lab: A customisable 3D platform for agent-based AI research
They specifically suggested "ALE/DMLab for DRL" but I couldn't find an obvious result for "ALE" when I searched for it. (It's a common acronym!)
I don't know of any courses specifically on testing ML (or numerical) code, but 'property-based testing' gives you great tools for testing code where coming up with input-output pairs is difficult. I wrote a paper on testing numerical or scientific code last year, and QuiviQ makes great PBT tools for Elixr.
↑ comment by Kenny · 2021-03-17T22:10:17.304Z · LW(p) · GW(p)
Your paper is excellent so far – very readable! Thanks again!
Replies from: Kenny↑ comment by Kenny · 2021-03-17T22:26:53.578Z · LW(p) · GW(p)
The whole paper was great – I'm sold on property testing!
Replies from: zac-hatfield-dodds↑ comment by Zac Hatfield-Dodds (zac-hatfield-dodds) · 2021-03-21T09:46:35.167Z · LW(p) · GW(p)
Thanks - I'm glad you liked it, and hope it helps!
1 comment
Comments sorted by top scores.