Skip to main content

Lab 8.6 - Stop a Service

Objectives​

By the end of this lab, you will be able to:

  • Identify a process using a specific port
  • Find the systemd service associated with a process
  • Understand why kill alone is not sufficient to stop a service
  • Use systemctl stop and disable to permanently stop a service
  • Automate the identification of a service via a bash script
  • Use advanced methods to diagnose services

Prerequisites​

  • Have a running Wiki.js site
  • Have nginx installed
  • Ubuntu 24.04 installed
  • Sudo/root access
  • Open terminal

Installing Nginx (if necessary)​

sudo apt update
sudo apt install nginx -y
sudo systemctl start nginx
sudo systemctl enable nginx
sudo systemctl status nginx

Context​

Mr. Eric, a system administrator, is trying to install and run an application named Wiki.js on an Ubuntu server. The application is supposed to listen on port 3000, but it is not working. After several unsuccessful attempts, Eric undertakes a series of investigations to identify and resolve the issue.

Step 1 – Checking the Port in Use​

Eric starts by checking if port 3000 is already used by another service. He uses the following command:

lsof -i :3000

Result obtained:

COMMAND   PID   USER     FD   TYPE DEVICE SIZE/OFF NODE NAME
node     9744   wikijs   22u  IPv4  59242      0t0  TCP *:3000 (LISTEN)

A node process is listening on port 3000. He notes the PID: 9744.

Analysis​

  • COMMAND: node - The process is a Node.js application
  • PID: 9744 - Process identifier
  • USER: wikijs - The user running the process
  • PORT: 3000 - The port in listening

Step 2 – Attempt to Free the Port with kill​

Eric attempts to manually stop the process:

# Attempt a graceful stop (SIGTERM)
kill -15 9744

# If that doesn't work, force stop (SIGKILL)
kill -9 9744

Verification​

After executing these commands, Eric checks again:

lsof -i :3000

Result:

COMMAND   PID   USER     FD   TYPE DEVICE SIZE/OFF NODE NAME
node     9842   wikijs   22u  IPv4  59243      0t0  TCP *:3000 (LISTEN)

Conclusion: The process is automatically restarted. A new PID (9842) appears. Another entity (probably a systemd service) is restarting it.

Why does the process restart?​

The process restarts automatically because it is managed by a systemd service with the Restart=always option in its configuration. Even if you kill the process, systemd immediately restarts it.

Step 3 – Searching for the systemd Service​

Eric suspects a systemd service. He therefore seeks to identify its exact name.

Commands Used​

# List all services
systemctl list-units --type=service

# List stopped services
systemctl list-units --type=service --state=exited

# List running services
systemctl list-units --type=service --state=running

# Filter to find Wiki services
systemctl list-units --type=service --state=running | grep -i wiki

Searching in Service Files​

# List service files by modification date
ls -l --sort=time /etc/systemd/system/

# Filter to find Wiki services
ls -l /etc/systemd/system/ | grep wiki

Cross Searches​

# Search in network connections
sudo ss | grep -i wiki
sudo ss | grep -i 3000

# Search Node.js processes
ps aux | grep node

# Search services containing "node"
systemctl | grep node

Winning Commands​

The most effective commands to identify the service are:

# Find running Wiki services
systemctl list-units --type=service --state=running | grep -i wiki

# Search Wiki service files
ls -l /etc/systemd/system/ | grep wiki

# View Node.js processes
ps aux | grep node

# Search services containing "node"
systemctl | grep node

Hypothesis confirmed: A service called wikijs.service seems to be responsible for the process.

Step 4 – Verification of the Correlation Between Process, Port, and Service​

Eric manually establishes the link between the node process, port 3000, and the service in question:

Identify the Process on the Port​

# View the process listening on port 3000
lsof -i :3000

# View all Node.js processes and their ports
sudo lsof -i -P -n | grep node

# View all Wiki.js processes
sudo lsof -i -P -n | grep wiki

Identify the Service File​

# Search for the service file
ls -l /etc/systemd/system/ | grep wiki

File identified: wikijs.service

Verify the Service​

# View the service status
sudo systemctl status wikijs.service

# View the service logs
sudo journalctl -u wikijs.service --no-pager --lines=50

Step 5 – Definitive Stopping of the Service and Port Release​

Eric properly stops the service and disables its automatic restart:

Stop the Service​

# Stop the service
sudo systemctl stop wikijs.service

Disable Automatic Restart​

# Disable the service at startup
sudo systemctl disable wikijs.service

Final Verification​

# Verify that the port is released
lsof -i :3000

# Verify the service status
sudo systemctl status wikijs.service

Result: Port 3000 is now released. The service will no longer restart automatically.

Bonus Step – Complete Scan of Open Ports (Optional)​

To ensure the overall state of the server, Eric uses the nmap tool:

# Install nmap
sudo apt install nmap -y

# Scan open ports on localhost
sudo nmap -sT -O localhost

# Scan a specific port
sudo nmap -p 3000 localhost

Reflection Questions​

  1. Why was port 3000 still blocked despite using kill?

    • Explain the role of systemd in service management
    • Why is kill alone not enough?

  2. What method allows identifying the name of the service using a specific port?

    • Describe the method step by step
    • Which commands are the most effective?

  3. What command could be used to search for all services containing the word wiki in their name or description?

    • Propose several approaches

  4. Propose a bash script to automate this investigation.

    • The script should automatically identify the service associated with a port

Appendix 1: More Advanced Methods​

The described method (manual search from the port with lsof, ps, systemctl, etc.) is reliable but artisanal. There are more advanced and automated methods, useful for complex environments, such as production servers or clusters.

1. systemd-cgls or systemd-cgtop​

These tools allow visualizing the hierarchy of services and processes controlled by systemd.

# View the complete hierarchy
systemd-cgls

# Filter for Wiki.js
systemd-cgls | grep -i wiki

# Filter for PostgreSQL
systemd-cgls | grep -i postgres

# Filter for Nginx
systemd-cgls | grep -i nginx

# View the hierarchy of a specific service
systemd-cgls /system.slice/wikijs.service

# List the cgroups
ls /sys/fs/cgroup/system.slice/ | grep wiki

# View Wiki processes
ps -ef | grep wiki

2. systemctl status Combined with --property​

You can query all services and request to see their MainPID, then cross-check with the port:

# View all MainPID of services
systemctl show --all --property=MainPID --no-pager

# For a specific service
systemctl show wikijs.service --property=MainPID

Then:

# View the PID using port 3000
sudo lsof -i :3000

And you compare the MainPID to the PID using the port.

3. Use fuser to Directly Kill the Process by Port​

# Kill the process associated with port 3000
sudo fuser -k 3000/tcp

Warning: This method does not disable the service, so it may restart. It is useful in a temporary script.

4. With ss --processes​

# View the process listening on port 3000
sudo ss -lptn 'sport = :3000'

This directly shows:

  • the port,
  • the protocol,
  • the process,
  • and sometimes the service name if well configured.

5. Use netstat with --program (if available)​

# Install net-tools if necessary
sudo apt update
sudo apt install net-tools

# View the process on port 3000
sudo netstat -tulnp | grep 3000

Allows directly identifying the program using a port. You can cross-check with systemctl:

# View process details
ps -p <PID> -o pid,cmd

6. Via systemd Logs​

If the service has started well by systemd, you can do:

# View the service logs
journalctl -u wikijs.service

# View logs in real-time
journalctl -u wikijs.service -f

# View the last lines
journalctl -u wikijs.service --no-pager --lines=50

This shows you the execution history, including the listened port.

Appendix 2: Automate the Identification of the Service​

Bash Script: Identify the systemd Service Behind a Port​

Create a file recherche_service_port.sh:

#!/bin/bash

# Prompt the user to enter the port number
read -p "Enter the port number (e.g., 3000): " PORT

# Step 1: Get the PID of the process listening on this port
PID=$(sudo lsof -i :$PORT -sTCP:LISTEN -t)

# Check if a PID was found
if [ -z "$PID" ]; then
    echo "No process seems to be listening on port $PORT."
    exit 1
fi

echo "Process detected on port $PORT: PID = $PID"

# Step 2: Display information about the process
echo "Command related to this PID:"
ps -p $PID -o pid,cmd

# Step 3: Search for the systemd service to which this PID belongs
echo ""
echo "Searching for the corresponding systemd service..."

# Method 1: Search via systemctl
SERVICE=$(systemctl | grep $PID | awk '{print $1}')

# Method 2: If not found, search in service files
if [ -z "$SERVICE" ]; then
    echo "Searching in service files..."
    # Search the PID in cgroups
    for service_file in /etc/systemd/system/*.service; do
        if [ -f "$service_file" ]; then
            service_name=$(basename "$service_file")
            # Check if the service is active and matches the PID
            main_pid=$(systemctl show "$service_name" --property=MainPID --value 2>/dev/null)
            if [ "$main_pid" = "$PID" ]; then
                SERVICE="$service_name"
                break
            fi
        fi
    done
fi

# Method 3: Search via cgroup
if [ -z "$SERVICE" ]; then
    cgroup=$(cat /proc/$PID/cgroup 2>/dev/null | grep system.slice | cut -d: -f3)
    if [ -n "$cgroup" ]; then
        SERVICE=$(basename "$cgroup" .service)
    fi
fi

if [ -z "$SERVICE" ]; then
    echo "No systemd service automatically found for PID $PID."
    echo "Manual verification possible with:"
    echo "  systemctl status <service_name>"
    echo "or search in /etc/systemd/system/"
    echo ""
    echo "Useful commands:"
    echo "  ps -p $PID -o pid,cmd"
    echo "  systemctl list-units --type=service | grep -i wiki"
    echo "  ls -l /etc/systemd/system/ | grep wiki"
else
    echo "Detected systemd service: $SERVICE"
    echo ""
    echo "Service status:"
    systemctl status $SERVICE --no-pager -l
    echo ""
    echo "To stop the service:"
    echo "  sudo systemctl stop $SERVICE"
    echo "  sudo systemctl disable $SERVICE"
fi

Usage​

  1. Save the script in a file:
nano recherche_service_port.sh
  1. Make it executable:
chmod +x recherche_service_port.sh
  1. Run it:
./recherche_service_port.sh

What You Have Learned​

  • How to link a port to a process (lsof)
  • How to trace back to the service via systemctl and grep
  • How to automate a network diagnostic in Bash
  • How to use cgroups to identify services

Summary of Important Commands​

CommandDescription
lsof -i :3000View the process listening on port 3000
kill -15 PIDGraceful stop of the process (SIGTERM)
kill -9 PIDForce stop of the process (SIGKILL)
systemctl list-units --type=service --state=runningList active services
systemctl stop wikijs.serviceStop a service
systemctl disable wikijs.serviceDisable a service at startup
systemctl status wikijs.serviceView the status of a service
journalctl -u wikijs.serviceView the logs of a service
sudo ss -lptn 'sport = :3000'View the process on a port with ss
sudo fuser -k 3000/tcpKill the process on a port

Final Verification​

Ensure you have succeeded in:

  • Identifying the process using port 3000
  • Understanding why kill alone is not enough
  • Finding the responsible systemd service
  • Properly stopping and disabling the service
  • Verifying that the port is released
  • Using advanced diagnostic methods
  • Creating and using the automation script
  • Answering the reflection questions

Next Steps​

  • Lab 8.1: Managing systemd Services (review)
  • Lab 8.5: Multiple Wiki.js Instances (creating multiple instances)
  • TP 3: systemd and Process Examination (assessment)

Lab created on: December 2024