Many of the papers now judged most original and significant rely on massive compute resources, usually beyond the financial reach of academia. So where does that leave academic research?
Last week, I travelled to the South of France (such a hardship) to attend deep learning conference ICLR (International Conference on Learning Representations*).
Lots to take away, but one thing struck me in particular — many of the papers deemed most significant relied on massive compute resource that is usually unavailable to academics. I wondered — what avenues remain open for compute-limited academic contribution? I try to answer that question below. (I also wondered whether this will soon be a moot question, when GAFA et al have finished recruiting all the academics!).
