If you drop in percentiles, you lose points - if you increase in percentiles, you gain points. Your points are not fixed and you can lose/gain them based on your current percentiles. Head to Portal -> [call of arms] on the bottom left -> scoreboard (bottom left icon) and you can see your score breakdown by hovering over each activity.