Process Management in Linux | System Administrator

Process management is an integral part of any modern-day operating system (OS). The OS must allocate resources to processes, enable processes to share and exchange information, protect the resources of each process from other processes and enable synchronization among processes. To meet these requirements, the OS must maintain a data structure for each process, which describes the state and resource ownership of that process, and which enables the OS to exert control over each process.

What are processes and threads??

A process is simply an instance of a running program.
A thread of execution is often regarded as the smallest unit of processing that a scheduler works on.
Processes include a set of resources such as open files and pending signals, internal kernel data, processor state, an address space, one or more threads of execution, and a data section containing global variables.
Threads are the units of execution within a program.

In Linux Operating system, the scheduling of thread or processes take place with a common concept of tasks–
A Linux Scheduler will schedule tasks and these tasks could be

1. a single-threaded process (e.g. created by fork without any thread library)
2. any thread inside a multi-threaded process (including its main thread)
3.kernel tasks, which are started internally in the kernel and stay in kernel space (e.g. kworke,kswapd etc…)

Each thread within a process has a
1. unique program counter,
2. process stack,
3. and set of processor registers

Threads are light weight. They don’t have their own memory spaces and other resources unlike processes. All processes start with a single thread.So they behave like lightweight processes but are always tied to a parent “thick” process. So, creating a new process is a slightly heavy task and involves allocating all these resources while creating a thread does not. Killing a process also involves releasing all these resources while a thread does not. However, killing a thread’s parent process releases all resources of the thread.

A process is suspended by itself and resumed by itself. Same with a thread but if a thread’s parent process is suspended then the threads are all suspended.In Linux, a process is created by means of the fork() system call, which creates a new process by duplicating an existing one.ssThe new process is an exact copy of the old process. (Now, a question might come immediately to our mind- who creates the first process??
The first process is the init process which is literally created from scratch during booting).The process that calls fork() is the parent, whereas the new process is the child. The parent resumes execution and the child starts execution at the same place, where the call returns. The fork() system call returns from the kernel twice once in the parent process and again in the newborn child.

The fork() function is used to create a new process by duplicating the existing process from which it is called.The existing process from which this function is called becomes the parent process and the newly created process becomes the child process.

Let us first see the standard definition of these system calls:

Fork : The fork call is used to duplicate the current process, the new process identical in almost every way except that it has its own PID.
The return value of the function fork distinguishes the two processes, zero is returned in the child and PID of child in parent process.Screenshot from 2017-10-13 18:26:39

Exec : The exec call is a way to basically replace the entire current process with a new program. The exec() call replaces a current process’ image with a new one (i.e. loads a new program within current process).
The new image is either regular executable binary file or a shell script.
There’s no a syscall under the name exec(). By exec() we usually refer to a family of calls:
int execl(char *path, char *arg, …);
int execv(char *path, char *argv[]);
int execle(char *path, char *arg, …, char *envp[]);
int execve(char *path, char *argv[], char *envp[]);
int execlp(char *file, char *arg, …);
int execvp(char *file, char *argv[]);

Here’s what l, v, e, and p mean:
l means an argument list,
v means an argument vector,
e means an environment vector, and
p means a search path.

Upon success, exec() never returns to the caller. If it does return, it means the call failed. Typical reasons are: non-existent file (bad path) or bad permissions.
Arguments passed via exec() appear in the argv[] of the main() function.
For more info: man 3 exec;
Example: exec_pid.c

Screenshot from 2017-10-13 18:31:44

Vfork : The basic difference between vfork and fork is that when a new process is created with vfork(), the parent process is temporarily suspended,
and the child process might borrow the parent’s address space. This strange state of affairs continues until the child process either exits, or calls
execve(), at which point the parent process continues.

This means that the child process of a vfork() must be careful to avoid unexpectedly modifying variables of the parent process. In particular,
the child process must not return from the function containing the vfork() call, and it must not call exit() (if it needs to exit, it
should use _exit(); actually, this is also true for the child of a normal fork()).

The intent of vfork was to eliminate the overhead of copying the whole process image if you only want to do an exec* in the child. Because exec*
replaces the whole image of the child process, there is no point in copying the image of the parent.

Clone : Clone, as fork, creates a new process. Unlike fork, these calls allow the child process to share parts of its execution context with the
calling process, such as the memory space, the table of file descriptors, and the table of signal handlers.

fork() and exec() Combined :

Often after doing fork() we want to load a new program into the child. E.g.: a shell.

Screenshot from 2017-10-13 18:35:40

The system() library function uses fork(2) to create a child process that executes the shell command specified in command using execl(3) as follows:
execl(“/bin/sh”, “sh”, “-c”, command, (char *) 0);

Each process on the system is in exactly one of five different states. This value is represented by one of five flags:

1.TASK_INTERRUPTIBLE: The process is sleeping (that is, it is blocked), waiting for some conditionto exist.
When this condition exists, the kernel sets the process’s state to TASK_RUNNING.
2.TASK_UNINTERRUPTIBLE: This state is identical to TASK_INTERRUPTIBLE except that it does not wake up and become runnable if it receives a signal.
This is used in situations where the process must wait without interruption or when the event is expected to occur quite quickly.
Because the task does not respond to signals in this state, TASK_UNINTERRUPTIBLE is less often used than TASK_INTERRUPTIBLE
3.TASK_ZOMBIE: The task has terminated, but its parent has not yet issued a wait4() system call.The task’s process descriptor must remain in case
the parent wants to access it. If the parent calls wait4(), the process descriptor is deallocated.
4.TASK_STOPPED: Process execution has stopped; the task is not running nor is it eligible to run.This occurs if the task receives the SIGSTOP,
SIGTSTP, SIGTTIN, or SIGTTOU signal or if it receives any signal while it is being debugged.
5.TASK_RUNNING: The process is runnable; it is either currently running or on a runqueue waiting to run .This is the only possible state for a process
executing in user-space; it can also apply to a process in kernel-space that is actively running.

Process Descriptor and the Task Structure:

– The kernel stores the list of processes in a circular doubly linked list called the task list.
– Process descriptor is nothing but each element of this task list of the type struct task_struct, which is defined in <linux/sched.h>.
The process descriptor contains all the information about a specific process.
– Some texts on operating system design call this list the task array. Because the Linux implementation is a linked list and not a static array,
it is called the task list.
– The task_struct is a relatively large data structure, at around 1.7 kilobytes on a 32-bit machine.
– This size, however, is quite small considering that the structure contains all the information that the kernel has and needs about a process.
– The process descriptor contains the data that describes the executing program open files, the process’s address space, pending signals,
the process’s state, and much more.
– This linked list is stored in kernel space.
– There is one more structure, thread_info which holds more architecture-specific data than the task_struct.

Allocating the Process Descriptor:

– Threads in Linux are treated as processes that just happen to share some resources. Each thread has its own thread_info. There are two basic reasons
why there are two such structures
1. thread_info is architecture dependent. task_struct is generic.
2. thread_info consumes the space of the kernel stack for that process, so it should be kept small.thread_info is placed at the bottom of the stack as a micro-optimization that makes itpossible to compute its address from the current stack pointer by rounding down by the stack size saving a CPU register.Each task’s thread_info structure is allocated at the end of its stack.The task element of the structure is a pointer to the task’s actual task_struct.

Process Descriptor Storage:

Processes are dynamic, so descriptors are kept in dynamic memory. n An 8KB memory area is allocated for each process, to hold process descriptor and kernel mode process stack.

Advantage:
Process descriptor pointer of current (running) process can be accessed quickly from stack pointer.
8KB memory area = 213 bytes.
Process descriptor pointer = esp with lower 13 bits masked.

Sending Signals To Processes:

The fundamental way of controlling processes in Linux is by sending signals to them.
There are multiple signals that you can send to a process, to view all the signals run
$ kill -l
1) SIGHUP 2) SIGINT 3) SIGQUIT 4) SIGILL
5) SIGTRAP 6) SIGABRT 7) SIGBUS 8) SIGFPE
9) SIGKILL 10) SIGUSR1 11) SIGSEGV 12) SIGUSR2
13) SIGPIPE 14) SIGALRM 15) SIGTERM 16) SIGSTKFLT
17) SIGCHLD 18) SIGCONT 19) SIGSTOP 20) SIGTSTP
21) SIGTTIN 22) SIGTTOU 23) SIGURG 24) SIGXCPU
25) SIGXFSZ 26) SIGVTALRM 27) SIGPROF 28) SIGWINCH
29) SIGIO 30) SIGPWR 31) SIGSYS 34) SIGRTMIN
35) SIGRTMIN+1 36) SIGRTMIN+2 37) SIGRTMIN+3 38) SIGRTMIN+4
39) SIGRTMIN+5 40) SIGRTMIN+6 41) SIGRTMIN+7 42) SIGRTMIN+8
43) SIGRTMIN+9 44) SIGRTMIN+10 45) SIGRTMIN+11 46) SIGRTMIN+12
47) SIGRTMIN+13 48) SIGRTMIN+14 49) SIGRTMIN+15 50) SIGRTMAX-14
51) SIGRTMAX-13 52) SIGRTMAX-12 53) SIGRTMAX-11 54) SIGRTMAX-10
55) SIGRTMAX-9 56) SIGRTMAX-8 57) SIGRTMAX-7 58) SIGRTMAX-6
59) SIGRTMAX-5 60) SIGRTMAX-4 61) SIGRTMAX-3 62) SIGRTMAX-2
63) SIGRTMAX-1 64) SIGRTMAX

And most signals are for internal use by the system, or for programmers when they write code.
The following are signals which are useful to a system user
SIGHUP 1: – sent to a process when its controlling terminal is closed.
SIGINT 2: – sent to a process by its controlling terminal when a user interrupts the process by pressing [Ctrl+C].
SIGQUIT 3: – sent to a process if the user sends a quit signal [Ctrl+D].
SIGKILL 9: – this signal immediately terminates (kills) a process and the process will not perform any clean-up
operations.
SIGTERM 15: – this a program termination signal (kill will send this by default).
SIGTSTP 20: – sent to a process by its controlling terminal to request it to stop (terminal stop);
initiated by the user pressing [Ctrl+Z].

In order to manage these processes, user should be able to

  • See all the processes that are running
  • View the system resources consumed by the processes
  • Locate a particular process and take specific action on it
  • Change the priority levels associated with processes
  • Kill the required processes
  • Restrict the system resources available to processes etc.

Linux offers many commands to the user to effectively handle the above mentioned scenarios.

1. ps

‘ps’ is one of the basic commands in Linux to view the processes on the system. It lists the running processes in a system along with other details such as process id, command, cpu usage, memory usage etc.  Some of the following options come handy to get more useful information

ps -a – List all the running / active processes

ps -ef |grep – List only the required process

ps -aux – Displays processes including those with no terminals(x)  Output is user oriented (u) with fields like USER, PID, %CPU, %MEM etc

2. pstree

In Linux, every process gets spawned by its parent process. This command helps visualize the processes by displaying a tree diagram of the processes showing the relationship between them.  If a pid is mentioned, the root of the tree will be the pid. Else it will be rooted at init.

3. top

‘top’ is a very useful command to monitor the system as it shows the system resources used by different processes. It gives a snapshot of the situation that the system is currently in. Its output includes data like process identification number(PID), user of the process, nice value, %CPU and %memory currently consumed by the process etc. One can use this output to figure out which process is hogging the CPU or memory.

4. htop

htop is similar to top, but is an interactive text mode process viewer.  It displays the per CPU usage and memory, swap usage using a text graph. One can use the Up/Down arrow key to select processes, F7 and F8 to change the priority and F9 to kill a process.   It is not present by default in the system and need to be installed explicitly.

5. nice

With the help of nice command, users can set or change the priorities of processes in Linux. Higher the priority of a process, more is the CPU time allocated by the kernel for it. By default, a process gets launched with priority 0. Process priority can be viewed using the top command output under the NI (nice value) column.

Values of process priority range from -20 to 19.  Lower the nice value, higher the priority.

nice <priority value> <process name> – starts the process by setting its priority to the given value

Screenshot from 2017-10-13 19:19:34.png

6. renice

It is similar to nice command.  Use this command to change the priority of an already running process.  Please note that users can change the priority of only the processes that they own.

Screenshot from 2017-10-13 19:20:55.png

Priority of process with id 3806 which had an initial priority of 0 is now changed to priority 4.

renice -u -g – change the priority of processes owned by the given user and group

7. kill

This is a command used to terminate processes by sending signals.  If a process is not responding to kill command, then it can be forcefully killed using the kill -9 command. But this needs to be used carefully as it does not give a chance for the process to clean up and might end up in corrupted files. If we are not aware of the PID of the process to be killed or want to mention the process name to be killed, then killall comes to rescue.

kill <pid>

kill -9 <pid>

killall -9 – kill all instances having the same process name

If you use kill, you need to know the process id of the process to be killed.   pkill is a similar command but can be used to kill processes using a pattern, i.e. process name,  process owner etc.

pkill <process name>

w:

w gives us information about the users who have currently logged in and the processes that they are running.  The header details displayed contain information like current time, how long the system has been running, total number of users logged in, load average of the system for the last 1, 5 and 15 minutes.

Understanding the Load Average Output:

From left to right, these numbers show you the average load over the last one minute, the last five minutes,
and the last fifteen minutes. In other words, the above output means:

load average over the last 1 minute: 1.05
load average over the last 5 minutes: 0.70
load average over the last 15 minutes: 5.09

over the last 1 minute: The computer was overloaded by 5% on average. On average, .05 processes were waiting
for the CPU. (1.05)

over the last 5 minutes: The CPU idled for 30% of the time. (0.70)

over the last 15 minutes: The computer was overloaded by 409% on average. On average, 4.09 processes were waiting
for the CPU. (5.09)

this load average is calculated by combining both the total number of process in the queue, and the total number of
processes in the uninterruptable task status.

You probably have a system with multiple CPUs or a multi-core CPU. The load average numbers work a bit differently
on such a system. For example, if you have a load average of 2 on a single-CPU system, this means your system was
overloaded by 100 percent — the entire period of time, one process was using the CPU while one other process was
waiting. On a system with two CPUs, this would be complete usage — two different processes were using two different
CPUs the entire time. On a system with four CPUs, this would be half usage — two processes were using two CPUs,
while two CPUs were sitting idle.

To understand the load average number, you need to know how many CPUs your system has. A load average of 6.03 would
indicate a system with a single CPU was massively overloaded, but it would be fine on a computer with 8 CPUs.

less /proc/cpuinfo | grep processor

OR

rupin@localhost:/proc$ nproc
4
rupin@localhost:/proc$

 10. pgrep

pgrep stands for “Process-ID Global Regular Expression Print”. It scans the currently running processes and lists the process IDs that match the selection criteria mentioned on command line to stdout. Useful for retrieving the process id of a process by using its name.

11. fg , bg

Sometimes, the commands that we execute take a long time to complete. In such situations, we can push the jobs to be executed in the background using ‘bg’ command and can be brought to the foreground with the ‘fg’ command.

We can list all the background processes using ‘jobs’ command

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s