I am a non resident.
I had some Australian source income some years back and lodged returns but have not lodged returns for last 5 years without a problem.
I will have some Australian source income in current year (20-21) so I will lodge a return.
As long as you are definitely a non resident (and not someone working overseas on a contract) and you have no Australian source income, you should be ok.