Chasing 268 for victory in the first Test, Australia were all out for 161, their innings including a record 154 consecutive deliveries without a run.
Australia - the world's top-ranked Test side - had a first-innings lead of 86.
But Kusal Mendis' superb 176 - his first Test century - and five wickets from Rangana Herath proved decisive.
Australia captain Steve Smith made a gritty 55 before tail-enders Steve O'Keefe and Peter Nevill resisted for 178 balls, scoring four.
The second Test in Galle starts on Thursday, 4 August.