There are plenty of programs with fewer students, less support from fans, a less plentiful local talent pool of players, and less to offer as an institution, yet many of them have much better football teams. The President of the school is the difference, it starts there, it ends there, and it's either there or it's not.
As an admitted outsider looking in, it appears that someone somewhere took note of this, as the facilities are being improved, and I think you have a good coaching staff. I think the key to the future of your program, will be the new President.
I do not have a depth chart or anything, but it sort of looks like you have a very young team. Is this not the case? I do not mean based on the ratio of the roster necessarily, but based on the best players you can put on the field for the future of the program. Let me ask you a question, if you were the head coach with 2 more years on your contract, would play the young guys for the future of your team in order to have a breakout season in 2 years, or would you play the best guys to have a mediocre season this year?