I'm having trouble getting the hang of thread(/process) prioritization on Linux, scheduling policy selection, what to choose when and how, and what the exact effects are.
Is there any documentation (like a guide) somewhere, preferably with concrete examples and timelines, which I could consult?
I'm having trouble getting the hang of thread(/process) prioritization on Linux, scheduling policy selection
The prioritization works by utilizing the underlying OS' thread and process priorities and it is hard to generalize about the specifics of it from the standpoint of documentation which may be why you've not found guides online.
My recommendation is (frankly) to not bother with thread priorities. I've done a large amount of threaded programming and I've never found the need to do anything but the default prioritization. About the only time thread prioritization will make a difference is if all of the threads are completely CPU bound and you want one task or another to get more cycles.
In addition, I'm pretty sure that under Linux at least, this isn't about preemption but more about run frequency. Many thread implementations use a priority scheduling queue so higher frequency threads get preference with logic in there to avoid starving the lower frequency threads. This means that any IO or other blocking operation is going to cause a lower priority thread to run and get its time slice.
This page is a good example of the complexities of the issue. To quote:
As can be seen, thread priorities 1-8 end up with a practically equal share of the CPU, whilst priorities 9 and 10 get a vastly greater share (though with essentially no difference between 9 and 10). The version tested was Java 6 Update 10. For what it's worth, I repeated the experiment on a dual core machine running Vista, and the shape of the resulting graph is the same. My best guess for the special behaviour of priorities 9 and 10 is that THREAD_PRIORITY_HIGHEST in a foreground window has just enough priority for certain other special treatment by the scheduler to kick in (for example, threads of internal priority 14 and above have their full quantum replenished after a wait, whereas lower priorities have them reduced by 1).
If you must use thread priorities then to you may have to write some test programs to understand how your architecture utilizes them.