I would say the day of workout and day after workout more important then day before. I would say Saturday is devour time and Sunday for pussification. However, why not maintain intake levels consistantly? You have a baseline calorie intake you need, and whatever you deem necessary for recovery and growth. Recovery and growth occur on Sunday too.
What are your stats and goals and what type of diet will lead you to that is the real question.