Cook rice according to package directions; set aside, and keep warm.
Preheat oven to 350°. Lightly grease a 2 1/2-quart casserole dish, or 6 individual serving dishes.
In a large skillet, melt butter over medium-high heat. Add onion, mushrooms, and celery; cook 5 to 7 minutes, stirring occasionally, or until tender. Add flour; cook 2 minutes, stirring constantly. Add milk, and cook, stirring frequently, until thickened, about 2 minutes. Remove from heat.
Stir in soup, chicken, red pepper, salt, pepper, and rice. Spoon into prepared baking dish; sprinkle evenly with cheese, and bake 30 minutes.