You definitely should work out more. What you should do is to go to the gym, at least 3 times a week. You shouldn't starve yourself. Instead you should find a healthy diet, with lots of vegetables and fruits. You should have a routine that helps your body get used to physical activities.
