There were such things as dwarf gods.
Dwarfs were not a naturally religious species, but in a world where pit props could crack without warning and pockets of fire damp could suddenly explode they'd seen the need for gods as the sort of supernatural equivalent of a hard hat. Besides, when you hit your thumb with an eight-pound hammer it's nice to be able to blaspheme.
It takes a very special and very strong-minded kind of atheist to jump up and down with their hand clasped under their other armpit and shout "Oh, random fluctuations-in-the-space-time-continuum!" or "Aaargh, primitive-and-outmoded-concept on a crutch!"