When I was getting my degree in Civil Engineering, there were two teachers in the department that had bad reputations to some and great reputations to others. My Analysis teacher and my Geotechnical teacher were probably the best teachers you could ever get in college. They knew their stuff DOWN and they could teach it brilliantly. BUT they were demanding and very hard. Of course, I took as many classes with these teachers that I could. Some students avoided them, and they aren't doing as well as I am out of college. I took the high road and already after 1.5 years, I'm in charge of the Geotech's brand new soils lab.
The point is this, Alexander. What is your reputation? Is it all bad, or are there students that sign up to take your classes because you teach well. If the latter is true, screw the other kids. They're going to be failures due to their own indolence. If it is the prior, look at how you are interacting with the students. Are you teaching the students well? You may know alot, but is it getting across. If it isn't, step back, toss away all your class notes and start over. Are you teaching out of a book, directly out of a book and nothing else. I hated that! When students ask questions, do you speak down to them or do you speak them up to the answer? Just some things to look at.
If you aren't getting your points across, the answer won't be to make the class easier! That will get larger classes, but make you feel depressed. Work on your relation with the class, not the difficulty of the homework. This is college for crying out loud. What were they expecting?