Re: [hackers] [quark][PATCH] Add a config switch to enable/disable NPROC limit

From: Laslo Hunhold <dev_AT_frign.de>
Date: Mon, 25 Jan 2021 16:46:27 +0100

On Mon, 25 Jan 2021 14:17:17 +0100
Giulio Picierro <giulio.picierro_AT_uniroma2.it> wrote:

Dear Giulio,

> sorry for the late reply, I had a really busy week (course to teach)
> :/.

don't worry about it! This is a mailing list and meant to be
asynchronous. Don't feel pressured to response and rather take your
time; it's a pastime after all.

> I didn't have the chance to test the new code, which I will do soon,
> hopefully.
>
> If I understand correctly your solution is to read the rlimit from
> the system and then update accordingly.
>
> Now it seems to me a good solution, however I have to ask: do we
> really need to set the rlimit on the number of processes (which are
> fixed)?
>
> I mean, what is the rationale behind it? Security purposes?
>
> I'm just asking out of curiosity, to understand better the design
> choices :D.

There are two aspects at play here, the limit on open file descriptors
of the current process (RLIMIT_NOFILE) and the limit on threads per user
(RLIMIT_NPROC).

The former is simple: It's a per-process-limit and we just apply a
heuristic and find a coarse upper bound for file descriptors the quark
process will consume. You see we set both cur and max, which is because
a program usually gets a signal when it exceeds the "soft" cur-limit,
but we don't want that. We want to either succeed or fail hard. The
nice thing about setrlimit() is that if the process is not privileged
(or has CAP_SYS_RESOURCE set) it can always set the soft limit and
irrevocably reduce the hard limit, so in case the limits are already
large enough, we don't even need CAP_SYS_RESOURCE.

The latter case is much more difficult, because the thread-limit is not
per-process but per-user. However, there isn't really a portable way to
find out how many current threads a user has. Say we give quark the
flag "-t 50" and are thus telling quark to spawn 50 serving-threads.
However, if the user has a thread-limit of 1000 and already has 990
threads active, pthread_create() will fail after creating 10 threads.
However, if the user has a thread-limit of 90 and only 20 active
threads, the thread creation will succeed (we suppose in both cases
that the number of "foreign" threads is constant).

Usually the thread limit is very high and not an issue, but there might
be configurations where that is not the case. The approach I worked out
for quark is kind of a hack, because what it does is just ask the
system to increase the thread-limit by the number of threads quark
needs. If that fails, e.g. if we are already at the kernel limit, we
just ignore that error, because that's as much as we can hope to do.

The only case where quark now could possibly fail with this heuristic
is on a system where a user is close to thread-saturation (in terms of
its limits, which must be way below the kernel limits) and some
user-program rapidly spawns threads in the few miliseconds between
quark's thread-limit-increase (triggering a TOCTOU) and
thread-allocation. However, even in this extreme case, quark would just
error out on pthread_create() and this is no security issue or
anything.

One could get rid of setting the thread limit by spawning the worker
threads before dropping root, but I just don't feel comfortable with
that. The earlier you lobotomize yourself, the better.

With best regards, hoping this was helpful to you

Laslo
Received on Mon Jan 25 2021 - 16:46:27 CET