The Weary Travelers
Paper Reviews
Paper reviews are more than just peer reviews. We aim to first provide a detailed summary of what’s in the paper, followed by an assessment of what impression it left behind, and how it fits with developments that have happened since it was published. We strive to keep the word count of the review less than that of the paper. 🛜
- 2023-11-05: : The transformers paper: Attention is all you need
- 2023-10-01: : The Batch Norm paper: Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- 2023-08-13: : The Adam paper: Adam: A Method for Stochastic Optimization
- 2023-07-15: : The ResNet paper: Deep Residual Learning for Image Recognition
- 2023-05-06: : The Hopfield Nets paper: Neural networks and physical systems with emergent collective computational abilities
- 2023-04-23: : The AlexNet paper: ImageNet Classification with Deep Convolutional Neural Networks
Outlines
Paper outlines are like reviews, but with less detail. We expect there to be more of them. Part of the reason we’re doing these is because we discovered that we each have a different approach to outlining. 🛜
- 2023-11-12: : MLPs at the PFLOP scale: Scaling MLPs: A Tale of Inductive Bias
- 2023-10-22: : BatchNorm and Loss Landscape: How does Batch Normalization help optimization?
- 2023-10-08: : The Layer Normalization paper: Layer Normalization
- 2023-09-17: : Sharp minima considered not so harmful: Sharp Minima Can Generalize For Deep Nets
- 2023-09-10: : The AMSGRAD paper: On the convergence of Adam and beyond
- 2023-08-19: : The AdamW paper: Decoupled Weight Decay Regularization
- 2023-07-30: : Mini-batch Bayesian learning: Bayesian Learning via Stochastic Gradient Langevin Dynamics
- 2023-07-23: : Batch Size vs. Learning Rate: Don’t Decay the Learning Rate, Increase the Batch Size
- 2023-06-25: : Loss landscape paper: Visualizing the Loss Landscape of Neural Nets
- 2023-04-29: : Suhail’s outline (Discriminative): A Case Against the Goto Statement
- 2023-04-29: : Chris’ outline (Generative): A Case Against the Goto Statement
Ideas
A collection of useful principles to make sense of the world. These are more traditional blog posts, each with a specific topic. Sometimes these will be ideas we’ve developed and hope to share more broadly, and sometimes just ideas already out there that we think could be explained better. 🛜
- 2023-11-19: : The Last Inch Problem: Sense and nonsense can be arbitrarily close together: The Last Inch Problem
- 2023-10-15: : Non-additive Loss functions: Why do loss functions have to be additive, anyway?
- 2023-07-09: : CPS: Continuation Passing Style
- 2023-06-11: : Faking it: Defunctionalization: Simulating higher-order functions
- 2023-06-03: : The Expression Problem: The challenge in writing extensible, modular code
- 2023-05-27: : The three E’s: The three E’s: Elegant, Efficient and Effective
- 2023-05-14: : Is-a vs Has-a: Inheritance vs Composition
How-Tos
A collection of how-to guides for specific tasks. Often, the greatest challenge is in putting ideas into practise. These guides aim to reduce the difficulty of this task. 🛜
- 2024-07-16: :
pdmandpdm-conda: How to do Python development without tears - 2023-09-03: :
org-babelas a superpower: How to overcome syntactic limitations inorg-mode - 2023-08-27: :
ox-publish: How to generate a website withorg-mode - 2023-08-06: :
ox-html: How to exportorg-modeto HTML - 2023-06-18: : Discord image generator bot: How to connect an image generating chat bot to a Discord server
- 2023-05-21: : Btrfs snapshots: How to effectively snapshot and revert with Btrfs
Miscellaneous
A collection of posts that couldn’t be better categorized elsewhere. Nothing is off-topic here. 🛜
- 2023-07-02: : Tool Review: Static site generators