Throttling HSDir connections

I have noticed that as soon as I get the HSDir flag, the number of times my relay complains about the computer not being fast enough to handle the connections increases dramatically. Is there a way of throttling the number of HSDir requests? Or preventing a relay from acquiring this flag?

Comment out dirport line in torrc?

That was what I immediately thought, but…

I don’t think a Dir port is required anymore? From version

Anyway, I don’t have a DirPort specified in my torrc - but I still have the HSDir flag.