What's a good way to test basic machine learning code?

post by Kenny · 2021-03-11T21:27:15.535Z · LW · GW · 1 comment

This is a question post.

Contents

  Answers
    3 Kenny
    2 Zac Hatfield Dodds
None
1 comment

I originally 'scoped' this question based on my inspiration, i.e. a set of example data and their expected answers for various machine learning algorithms or their code components. In other words, a set of test cases that could be used to test implementations of those algorithms or components.

I took the Coursera course Machine Learning (by { Stanford University / Andrew Ng}) and it was great!

I took it because I wanted some 'finger practice' of writing machine learning code and actually running it on my own computer. The course was perfect for that!

One thing in particular that was nice is that the exercises/quizzes/tests that required writing code would have some kind of automated 'endpoint' whereby you could submit your code for testing. (That kind of 'test kata' practice is a really fun way to learn any kind of programming topic, e.g. a new programming language.)

I found these links, on the above course's forum on Coursera, linking to some of the info about the exercises:

The course uses Octave, which was fine.

But I want to basically repeat the course – but using completely different tools. (I want to play with this library in particular.)

What are the best alternatives to the course I took, or similar resources that would be helpful in implementing basic machine learning algorithms, but with arbitrary programming tools?

Answers

answer by Kenny · 2021-03-17T22:12:18.773Z · LW(p) · GW(p)

Someone suggested these resources:

They specifically suggested "ALE/DMLab for DRL" but I couldn't find an obvious result for "ALE" when I searched for it. (It's a common acronym!)

comment by gwern · 2021-03-18T00:28:16.286Z · LW(p) · GW(p)

ALE is doubtless the Atari Learning Environment. I've never seen an 'ALE' in DRL discussions which refers to something else.

Replies from: Kenny
comment by Kenny · 2021-03-18T00:54:38.029Z · LW(p) · GW(p)

Thanks! Of course you would know :)

answer by Zac Hatfield-Dodds (Zac Hatfield Dodds) · 2021-03-14T03:22:21.303Z · LW(p) · GW(p)

I don't know of any courses specifically on testing ML (or numerical) code, but 'property-based testing' gives you great tools for testing code where coming up with input-output pairs is difficult. I wrote a paper on testing numerical or scientific code last year, and QuiviQ makes great PBT tools for Elixr.

comment by Kenny · 2021-03-14T19:26:30.123Z · LW(p) · GW(p)

I see now that my question title could be better. I'm more looking for test cases, than testing tools.

I've added your paper to my reading queue! Thanks!

comment by Kenny · 2021-03-17T22:10:17.304Z · LW(p) · GW(p)

Your paper is excellent so far – very readable! Thanks again!

Replies from: Kenny
comment by Kenny · 2021-03-17T22:26:53.578Z · LW(p) · GW(p)

The whole paper was great – I'm sold on property testing!

Replies from: zac-hatfield-dodds
comment by Zac Hatfield-Dodds (zac-hatfield-dodds) · 2021-03-21T09:46:35.167Z · LW(p) · GW(p)

Thanks - I'm glad you liked it, and hope it helps!

1 comment

Comments sorted by top scores.

comment by Kenny · 2021-03-14T19:29:01.709Z · LW(p) · GW(p)

I started working on re-implementing my Octave code for the course I took previously and it's going pretty well!