As my research seems to be referenced in many journals and application areas, I get many requests for reviewing papers (maybe an average of once a week). As I am not a full-time researcher, I obviously do not have much time to spend on reviewing papers, especially if the unrealistic deadline of something like 4 or 6 weeks is asked for.
Journal editors: please keep in mind that anyone active in research gets lots of refereeing requests and any manuscripts that are accepted for refereeing joins the refereeing queue. In my case, my turnaround time is around 10 weeks to 3 months. So please don't bother to ask me to review a paper unless you have a more realistic timeline. I would be interested in seeing how some of my theory is used in different disciplines if there is sufficient time.
During teaching terms, my priority is teaching and giving feedback to (undergraduate and research) students; I might not be able to do refereeing until the end of the term; maybe I can do refereeing in one block of time in the middle of the term.
I only consider reviewing manuscripts with a reasonable deadline of around 10 to 12 weeks. Note that I am also an associate editor for two journals and I use a deadline of around 3 months. Even with this deadline, I sometimes have trouble finding referees who can agree to and make the deadline.
Have you kept track of how many of your reviewers make your deadline, and whether you get good refereeing reports from those who agree to such short deadlines. Personally, I think the quality of published papers in journals has been declining, maybe partly to poor refereeing efforts with short deadlines.