This may be a stupid question, but do you warm up your spindle before cutting?
A friend told me that I have to let it run for 5-10 min. before cutting. It helps with precision and spindle collet expansion (bit falling out). It has happened once or twice that my bit has fallen out during long runs, but I just assumed I had not tightened it enough.